If you know in your heart that you're not hurting or stepping on anyone, then keep doing what makes your heart happy. No one has the right to stop you from pursuing what you want in life.
Always remember to do what helps you grow and become a better version of yourself. Never let anyone bring you down or make you feel less than you are. Know your worth, stand your ground, and keep moving forward.