Huh, I'll experiment a bit later when I'm on a computrr. But off the top of my head,, I can think of two ways that might work.
Rather than cleaning the dots off, you should try selecting the black lines and copy them to a new layer. Either with the magic wand tool, or select them by doing:
(Top toolbar) select>select color range>click on the black outlines.
That or brushing over them manually which would be a pain in the butt.
Can't actually try these for you since I'm on my phone lol.