In a large bowl or in the food processor stir together the flours, rolled barley, sugar, and salt. Cut in the butter until the mixture resembles coarse meal. Add enough of the milk to form a dough that will hold together in a ball.
Divide the dough into 2 equal portions for rolling. On a floured surface roll out to 1/8 to 1/4 inch. Cut into 2-inch circles or squares and place on a lightly greased or parchment-lined baking sheet. Pierce each cracker in 2 or 3 places with the tines of a fork.
Bake for 25 to 30 minutes, or until medium brown. Cool on a wire rack.