No, it could not fly much faster with the available energy.
Lift is a question of wing area and dynamic pressure. Solar Impulse 2 has 269.5 m² wing area to carry its 2.3 tons of mass. This is a wing loading of just 8.53 kg/m²; much less than even gliders have (they start at around 30 kg/m²). This allows it to fly very slowly; if we assume it flies at the lift coefficient $c_L$ for optimum range for propeller aircraft
$$c_L = \sqrt{\pi \cdot AR \cdot \epsilon \cdot c_{D0}}$$
and guess the zero-lift drag coefficient $c_{D0}$ to be 0.029 and the aspect ratio $AR$ to be 19.18, the result is 1.377. This is already close to the maximum lift coefficient, which I expect to be 1.6. Now we need the lift equation to calculate the speed which is needed to keep it flying at this lift coefficient:$$v = \sqrt{\frac{2\cdot m\cdot g}{c_L\cdot S\cdot\rho}}$$
Let's assume it flies in 9000 m altitude where air density $\rho$ is only 0.4671 kg/m³, the result is 16.13 m/s or 58 km/h. Now consider that it will fly at the beginning, late at night and at the end of the flight at lower altitudes, and may not be able to fly a straight course (maybe to fly around bad weather), and the 55 km/h look entirely plausible, if not outright speedy. At sea level, the optimum range speed is only 9.96 m/s or 35.86 km/h, just about its take-off airspeed.
Flying faster would need more energy and would not allow Solar Impulse 2 to recharge its batteries for the night hours. Solar flight must be slow to be possible at all.