This have nothing to do with Singleton pattern.
What you need is some kind of resource manager type that would register factories objects, one for each resource type you want it to be able to provide. Each factory object would know how to load the associated resource type.
Then when you want a resource, you use a function to get the resource from a resource manager instance, that will find the the resource already loaded for you or will request the resource-type-specific factory it owns to load it.
It's interface would look like this (but it really depends on your context in the end):
void add_factory( std::shared_ptr<ResourceFactory<ResourceType>> factory );
std::shared_ptr<ResourceType> get(); // will look into registered factories and provide a resource instance if it
This is classical way to manage resources gathered in an object.
Once you have such type (a ResourceManager class for example) working, it's easy to make it (or not, as others will suggest) Singleton. I don't recommend it though.
It's not your class that is polymorphic, it's the resource type.
A better way, most of the time, would be to pass a reference to the resource manager instance into functions and types that need it. There are cases where making it a singleton would be better, but it's very rare.