Quoted By:
I've found that it helps to live in a big city. I lived in Orlando up until August when I moved to Chicago.
Orlando had a fairly small downtown which was mainly offices and corporate property with not many people out, compared to Chicago which is a bustling metropolis. Although Orlando was much more laid back than Chicago, most of the people you would meet in Orlando were a lot more hostile or uneasy towards cameras (Had a dudebro track me down after he walked into the frame of a shot I never took and threatened to smash my XA if I wasn't in a public space with other people like I was).
Chicago is a lot different in that there's a lot people in a rush to do get somewhere, so they could care less about that guy walking down the crowded street with his camera. If you do ever piss someone off, it'll probably be a foreign tourist; I've noticed more nasty looks from Indian and southern Asian tourists than any other group when downtown.
Don't get in people's faces with a flash like you're Eric Kim, don't loiter around people and act like a creep, take two shots at max, look like a decent member of society, don't act sheepish or withdrawn, know when and when not to take a photo, and always be respectful. You're just a normal dude who likes street photography, not a paparazzo or a weird creep.