You are correct in both regards -- as Terry's comment points out, an undisturbed pressure altimeter's reading (imagine one sitting on a bench outside the local hangar) will change with the local barometric pressure. TSO'ed "sensitive altimeters" (I'm not sure if a non-sensitive altimeter even exists any longer, although there was a point in time when they did!) are required to have a knob and window, called a Kollsman window, for setting the local barometric pressure, or standard pressure (29.92) when above the transition altitude (18,000' in the US, check with the local aviation regulator for where you are in the world). "Glass cockpit" systems have a knob/function for the same job -- whether it be a dedicated knob, or a mode on a multi-function knob.

You are correct in both regards -- as Terry's comment points out, an undisturbed pressure altimeter's reading (imagine one sitting on a bench outside the local hangar) will change with the local barometric pressure. TSO'ed "sensitive altimeters", i.e. ones with a hundreds pointer, not just a thousands, are required to have a knob and window, called a Kollsman window, for setting the local barometric pressure, or standard pressure (29.92) when above the transition altitude (18,000' in the US, check with the local aviation regulator for where you are in the world). "Glass cockpit" systems have a knob/function for the same job -- whether it be a dedicated knob, or a mode on a multi-function knob.