I wonder this because I have seen pictures of ww1 airplanes that have rudder much smaller than today, but when did the first rudder come out.
Planes as early as the wright glider (1902) had a rudder, it appears from pictures that the 1900 and 1901 variants did not have rudders but its hard to tell from the photos.
Depending on your definition if you are talking about powered flight the Wright Flyer (1903) had 2 rudders much like its earlier glider predecessor.
In the lighter than air category, the Giffard Dirigible (1852) had a sail like rudder in the aft of the ship. So that may take the cake if you have consider it a true rudder.
Side Note: Patented in 1930 the V-Tail design could be argued as the first design that did not (as far as I know) employ a rudder. The earlier Beechcraft Bonanza's are the most common plane of that design in commercial production. There is quite a bit of debate as to the safety of ruddervators and the general theme seems to be that they are not perfect and do require some care to fly properly.
The Wright Flyer had two rudders. That flew in 1903. Some of the wright bros. earlier gliders also had rudders ... To test the technology and to determine the optimum size.
"[The Wrights] could fly straight ahead well enough, but when the brothers rolled into a turn the high wing would create more drag than the low wing. The imbalance in drag made the glider yaw in the opposite direction of the turn. After a few years of experiments, the Wrights finally hit upon the idea of using a rudder to counteract the drag. The rudder added aerodynamic yaw control and their 1902 glider was to first machine ever with three-axis control — roll, pitch and yaw."
According to this site, the need for a rudder comes to counter adverse yaw, which is still the main purpose of the rudder.
Rudder appears in 1902.