If museums are on your list of things to see in Withernsea, then you might want to head to RAF Holmpton. If you want to see more of the museums in Withernsea, Withernsea Lighthouse Museum isn't far either.
St Nicholas Church is just one of the places to see in Withernsea, so why not enjoy the other things to explore during your travels? Take in the museums while you're in the area.