After some research, I've been able to highlight such a "bad result" (but really better than PDN) in Photoshop. It means they probably developped extremly complicated algorithms based on integers.
As such an algorithm is certainly by far too complicated to me, I think the good choice for PDN would be to keep using floats but without any rounding. Hue, saturation and lightness must be floats and remain floats to get modified until they turn back to RGB. For exemple, the Hue of RGB (85, 120, 201) is 221.8965. If you keep this exact value, you will get the exact RGB when you go back to RGB. If you round, ceil, floor or just cast this value, you won't be sure to get the exact value.
So, the only thing you have to do is to keep HSL as floats and the result will be color perfect. I tried it with all possible values for RGB (RGB => HSL => RGB) and the result is always exactly the same.