I know that officially distributed node binaries don't use system icu (as it's not guaranteed) but things could be seen another way with containers.
Why? especially to avoid full-icu install pain (it's been a while that it's not working well and no real solution still emerging).
In depth, I suggest to add the following lines when compiling node (here the alpine variant) until things get fixed into full-icu package: