"Thank God I only had to do one year in that environment."
So it sounds like you just lived in and went to a sh*tty part of town and school. That was where you lacked (class) privilege. Do you honestly think it was a utopia for your fellow black classmates?
the point is, you got out, and I'm guessing when you did you were no longer a minority and things went on to be hunkey dorey for the most part. The irony is, in some ways, even with the vast improvement in circumstances, things would in several ways be worse if a black student had switched up the way you did.
Hence white privilege. When you "better your circumstances" it puts you into perfectly neutral territory. When black people do, we add a negative. Just ask any rich black man driving around Beverly Hills in a luxury car at night.