I'm building an ultralight and it weighs 100 pounds empty. My only engine is 3HP. Is that good enough?
In general, regardless of the characteristics of your aircraft, 3 HP isn't enough power to lift the weight of the pilot safely. Your ultralight would be effectively a power-assisted glider, unable to climb in calm air. You'd have to launch like a hang glider (from a slope or cliff, or by one or another form of tow), though if your wing and layout are efficient it's likely you could stay up until you run out of fuel in calm air with careful, conservative flying.
Go look up some of the earliest, 1970s and 1980s vintage powered hang glider attempts. Some tried installing engines in that range. None kept at it, because very poor climb is an extremely unsafe condition.
It depends on the lift-to-drag ratio, and the lift versus airspeed characteristic of your wing. If you know the total weight of the plane with you in it, then you know how fast you need to fly to generate an amount of lift sufficient to exceed the dead weight of the plane- in the absence of drag. Then knowing the L/D ratio, you can solve for the drag force generated at that airspeed- which the engine must furnish- and which will subtract from the thrust generated by the engine. You then iterate the process to converge on the horsepower needed to propel the plane fast enough to lift off.