Wrexham only avoided relegation on the last day of the season last term, and Carey said: "Obviously I'm disappointed.
"It took us a long time to get going but we did create some chances.
"The crowd are getting on the players' backs and that doesn't help, but we are all frustrated.
"We'll keep working on it. You can either go home and hide your head under the pillow or you can just get on with it."
Rotherham boss Mark Robins hailed striker Andy Todd's development and said he "provided the quality" in the win.
He said: "Andy Todd has been threatening to produce things like that for us for a while.
"He's learning and he's getting into the right areas but often he takes just one touch too many.
"He provided the quality and overall it's a crucial victory to come here and take three points."
Sporting Life