More formally, a Markov Model describes a process as a collection of states with transitions between them.
Each of the transitions has an associated probability.
The next state in the process depends solely on the current state and the transition probabilities.
In a Hidden Markov Model, each state has a set of possible outputs that can be generated.
As with the transitions, each output also has a probability associated with it.