Probably the cheapest option tooday is Garmin eTrex. You can have one from Amazon for $90 with free shipping. I have one and it's very nice unit.
It doesn't have any maps or ploter so you will need paper charts.
It gives you all you need: position, bearing and distance to the waypoint, course and speed and even plot tiny map with your past track and waypoints. The screen has backlight but the letters on the screen are not the biggest though.
More important, it is waterproof and run on two AAA batteries. I find it tough and nicely build.
If budget is your priority I think this is the way to go, but consider additional expenses for charts.