Question about my bikes rear brake?

When I take my bike for a ride, I can hear a slight rubbing sound from the rear brake rubbing one the wheel. There is no resistance I can feel, but do I need to realign the brake? Or could that mean the wheel may be wobbling?