You could view a battery as a kind of pump. But instead of pumping water through pipes, the battery moves electrons through a wire (and through the things that the wire is connected to).
Here's how a battery works (the kind you buy at the checkout counter):
Chemical reactions within the battery cause electrons to be stripped away from the carbon electrode. Electrons will try to flow from a battery's negative terminal to its positive one, if allowed.
The battery is made up of a zinc can, which acts as the battery's container (although it's usually covered by a shiny metal casing), and a carbon rod, which is at the battery's center, suspended in a pasty mixture that, in an alkaline cell, contains potassium hydroxide.
A chemical reaction within the pasty mixture strips electrons from some of its atoms. These excess electrons collect on the zinc can, which acts as the negative terminal.
At the carbon rod are atoms with a shortage of electrons.
The electrons at the negative terminal want to go to positive terminal, they just need a way to get there. In our light bulb circuit, the way to get there is through the wire. The number of the electrons the battery can push through the circuit will depend on the resistance at the bulb's filament.
Because the electrons flow in one direction only, batteries produce direct current.
With Edison's direct current system, electricity was produced not by batteries but by a DC generator. The generator actually produced alternating current, which was then converted to direct current with a commutator.