BALTIMORE (AP) – The portrait used to hang in the hallway, welcoming children and parents to the Archbishop Borders School in Baltimore: a smiling Dr. Ben Carson in surgical scrubs, rubbing together the careful, steady hands that helped him become the nation’s most famous black doctor.

“The person who has the most to do with your success is you,” it reads.

That was before Carson’s presidential bid, before he withdrew from the race and endorsed Donald Trump, and before he was tapped to run the Department of Housing and Urban Development. It was before the president failed to condemn white supremacists who marched in Charlottesville, Virginia. And before Carson pushed policies critics say walk back civil rights protections for those living in subsidized housing.

“I took it down,” said Principal Alicia Freeman of the portrait she’s since moved from the school’s second floor hallway to a less visible spot inside a reading room bearing Carson’s name. The doctor’s inspirational message now feels hostile, she said.

“He was starting to become offensive.”

Carson’s story of climbing out of poverty to become a world-renowned surgeon was once ubiquitous in Baltimore, where Carson made his name. In some schools his memoir was required reading, an illustration of the power of perseverance. For a working-class, majority African American city wracked by racial division and neglect – where a baby born in a wealthy white neighborhood is expected to live two decades longer than one in a poor black area _ Ben Carson was hope.

But his role in the Trump administration has added a complicated epilogue, leaving many who admired him feeling betrayed, unable to separate him from the politics of a president widely rejected by AfricanAmericans here. In the last presidential nearly 85 percent of city voters cast ballots for Hillary Clinton.

“The Trump virus is weakening Ben Carson’s image,” said Bishop Frank Reid, a former pastor at Baltimore’s Bethel AME Church who met Carson at Yale, where both received their bachelor’s degrees. Carson is still respected, Reid said. “But he is no longer the hero he once was.”

Carson declined to be interviewed for this story. Instead, he sent a written statement.

“I understand what it means to be poor because I grew up poor,” the statement said. “I was fortunate to have my mother who was my compass – always steering me on course, helping me to see beyond our circumstances. That’s what I hope to do for the millions of low-income families HUD serves.”

Carson was born in Detroit, but Baltimore is the city that claims him. He rose to fame for his election, groundbreaking surgeries at Johns Hopkins Hospital and launched his scholarship program here. Carson would sometimes arrange for high school students to visit the National Great Blacks in Wax Museum on the city’s east side, where a figure in his likeness stands glossy and smiling in a white lab coat and stethoscope, in the middle of the Famous Marylanders display.

“This young AfricanAmerican who grew up in poverty and could have been put in jail or suspended from school made something great of himself. It was the American Dream,” Reid said. “We believed he could walk on water.”

Emmanuel Williams, 33, grew up in Northwest Baltimore. He learned about Carson in elementary school, a few years after Carson grabbed headlines for successfully separating conjoined twins attached at the skull.

“He was taught during Black History Month,” Williams said. “And everyone was so proud because it was happening here in Baltimore. It was mythic.”

“Sometimes I think the country looks down on us,”

he said. “So to have such a brilliant person who’s making history and making these great medical advancements in Baltimore? He was our crown jewel, and he was here.”

Since taking the reins at HUD, Carson has proposed sweeping rent increases for the poorest subsidized housing tenants, and has begun dismantling key Obama-era regulations designed to address racial segregation. Carson has considered stripping antidiscrimination language from the department’s mission statement, and voiced strong support for implementing work requirements for housing aid recipients. In a radio interview shortly after being confirmed, Carson said poverty “is a state of mind.”

Now, Williams said, “people feel betrayed.”

“He can’t come back from this,” he said.

The seeds of Carson’s approach to policy are scattered throughout his memoir. He has long promoted self-sufficiency and enthusiastically embraced the bootstrap ideology popular with conservatives.

“Success is determined not by whether or not you face obstacles, but by your reaction to them,”

Carson wrote.

But those messages, now coming from a politician, are being received differently.

“There’s a certain consistency to his message, it’s just the language is now different,” said Kurt Schmoke, Baltimore’s first elected African American mayor. Carson and Schmoke, now the president of the University of Baltimore, have been friends for decades.

“It’s more political, more partisan, and in my view, it’s harsher,” he said.

Schmoke said Carson’s achievements and philanthropic work haven’t been entirely eclipsed by his association with the Trump administration. But Carson’s political turn has changed the way many people see him.

“You can’t take away the fact that he’s done outstanding things for people throughout his life, that can’t be erased,”

Schmoke said. “But I do think there’s clearly more people who view him through a political lens and that affects how he’s viewed in this community.”

Shaun Verma, a Ben Carson Scholarship recipient from Georgia, says Carson’s use of his story of hard work and determination to justify scaling back the safety net for the same communities that raised and revered him “is really disappointing.”