Widgets are WordPress' name for the modules in your site's sidebar. This will vary depending on your theme, but typically these are viewable on the right or left side of your pages and posts:
To customize what appears here, go to Appearance > Widgets in your admin dashboard's sidebar. There will be at least two sections available to you, "Available Widgets" and "Sidebar":
Available widgets are all the widgets you can use that aren't currently enabled. Just drag them over to the "Sidebar" box to add them to your site. Likewise, drag items out of the "Sidebar" box to remove them from your site.
The Text widget is the most versatile widget available for use, as it allows you to specify any text you want in your sidebar, for example a list of links to other important sites. It also lets you use some basic HTML, such as embedding photos, iFrames, and links.
You can also use the Navigation Menu widget to show a custom list of links. The widget will allow you to choose a menu you've already created.
Some themes may include various other widget locations besides "Sidebar", such as footer widgets, which you can experiment with.