Though the purists in street photography will tell you that you need to get up close and personal to someone on the streets, there isn't anything inherently wrong with using an 85mm lens. Of course, any wrongdoing comes with the intentions of the photographer.
In this case, your intentions should be to try to get a different look at the scene and capture it as it happens with lesser chance of being seen. If you feel like calling it such, you can think of it as being a fly on the wall. It's a term that so many street photographers aspire to be when they shoot. . thephoblographer.com