I'd agree with the other response that tepid is the best word for this, but if you don't like that one you could use chilly, chill, brisk, or even just the word cool. The word cool implies on its own something that's just slightly cold.
Body temperature > room temperature. Lukewarm/tepid kinda occupies the space between. It is technically warmer than its surroundings, but does not provide a substantial warming effect to the body.
Lukewarm is used almost exclusively for water, whereas room temperature is a reference to air temperature (either the current or a desirable one) Water and air exchange heat with the human body in different ways and at different rates. Room temperature air is fairly neutral to the body, but a 68F/20C swimming pool is rather chilly, and a 90F/32C room is not what I would call lukewarm.
Warm & cool both have an implication of comfort whereas hot & cold have more an implication of danger or discomfort. Maybe there is something to thinking about these on more than one axis: relative temperature vs desirability or pleasantness.
Context is weird. For things that are supposed to be “hot”, either “cool” or “cold could mean room temperature, above room temperature, but also not quite “warm”, or hotter than “warm” but below a target, expected, or usable temperature.