First of all, let's talk about the basic things you need for any hydro system. If you want to start with a super basic setup, try Deep Water Culture or Wick. These systems have no moving parts -- this means no pumps, no filters, no electricity needed. All you need is a reservoir(which you can easily DIY out of materials found at home, check out the link to find out how), a grow tray, growing medium, nutrient solution, pH testing kit (only a few dollars for strip tests), lights if your system is indoors, plants and of course, water.
If you decide you like hydroponics and want to move up to a more complex system, you can easily create a Nutrient Film system or even Ebb and Flow with a few simple add ons. You'll need to update the grow tray to fit whichever system you choose and purchase a pump and timer to circulate the water. None of these things cost too much money, and you can always keep your system to a size that is manageable for your budget.