The farthest known galaxies are extremely bright galaxies, much brighter than our own Milky Way. They have to be bright, otherwise we could not see them at that distance. We think there are other galaxies near them that are dimmer, but we cannot detect those dim galaxies due to the limitations of our telescopes (among other factors).
The most distant spectroscopically-confirmed galaxy known as of this writing (July 18, 2015) is EGSY-2008532660, which is at a redshift of 8.68 (here's a link to the scientific paper). A redshift of 8.68 means that this galaxy is about 30 billion light years away. Alternatively, we are seeing this galaxy as it existed only 0.6 billion years after the Big Bang, which makes it very young considering the Universe is only 13.7 billion years old.
Redshifts are one of the units Astronomers use to talk about distances to very far galaxies—the bigger the redshift, the further away the galaxy. Redshift is basically a measure of how stretched out in wavelength a galaxy's emitted light gets due to the expansion of the Universe (i.e., the distant galaxy emits bluer light that gets stretched into redder light which we observe). Astronomers like to use redshifts instead of other distance units (like parsecs or lightyears) because redshift is an unchanging measurement while the conversion between redshift and other distances requires assuming a specific cosmology (i.e., the current rate of expansion of the Universe, the amount of dark matter and dark energy, etc.) which involves numbers that are constantly updated by modern experiements. You can read about all the different types of distances in Astronomy here.
A spectroscopically measured redshift is basically the "gold standard" of distance measurements. There are other galaxies that are suspected to be at higher redshifts (between redshifts of 9 and 11) based on their observed colors, but there is a chance that those systems are just bizzare nearby things, and there are significant uncertainties associated with color-based redshifts. We generally refer to those systems as "candidate galaxies" until they can be spectroscopically confirmed. Getting good spectra for things so far away is tricky (they're very dim!), so there are a lot more high-redshift candidates than there are spectroscopically confimed high-redshift galaxies.
Looking out in distance is the same as looking back in time (because the light has taken a long time to get here), and Astronomers think that the galaxies we could see at redshifts of about 10 and bigger should be just forming. Before that (or further away) there probably weren't any galaxies yet.
The furthest galaxy you can see with your own eyes (without needing help from a telescope) is the Andromeda galaxy. It's so close that it really has the same redshift as us (0), but is actually about 2 million light years away.
This page was updated by Chelsea Sharon on July 18, 2015.