Covariance with Smart Pointers

Covariance can be a useful concept, e.g. when implementing the abstract factory design pattern. However, in modern C++, we should return smart pointers that are not recognized to be covariant like raw pointers by the compiler.

Abstract factory

I will not go into too much detail about the abstract factory pattern, as it is not the point of this article. You can best look it up in the “Gang of Four” book or on the web. For the code on this post I will borrow the very popular example also used on Wikipedia:

Consider an abstract factory returning equally abstract widgets for our GUI. Those widgets can be buttons, text fields, dropdown boxes etc. Depending on the GUI framework you use (e.g. differing by operating system), a concrete factory creates concrete implementations of the widgets.

Abstract factory with smart pointers

I have written about factories returning smart pointers earlier. For this example I’ll take the simpler alternative and use `std::unique_ptr`. Our basic code may look roughly like this:

#include <iostream>
#include <memory>

struct AbstractButton {
  virtual void click() = 0;
  virtual ~AbstractButton() = default;
};
struct AbstractWidgetFactory {
  virtual std::unique_ptr<AbstractButton> createButton() const = 0;
};

struct FancyButton : AbstractButton {
  void click() final override { std::cout << "You clicked me, I'm so fancy!\n"; }
};
struct FancyWidgetFactory : AbstractWidgetFactory {
  std::unique_ptr<AbstractButton> createButton() const final override {
    return std::make_unique<FancyButton>();  
  }
};

int main() {
  std::shared_ptr<AbstractWidgetFactory> theWidgetFactory = std::make_shared<FancyWidgetFactory>();
   
  auto theButton = theWidgetFactory->createButton();
  theButton->click();
}

A need for covariance

Let’s assume that our factories get some more features. For example, we could have a functionality that creates a simple message window with an “OK” button.

std::unique_ptr<AbstractWindow> createMessageWindow(std::string const& text) {
  auto theWindow = theWidgetFactory->createWindow();
  theWindow->addText(text);
 
  auto theButton = theWidgetFactory->createButton();
  theButton->setText("OK");
  theWindow->add(std::move(theButton));
  return theWindow;  
}

This is pretty abstract, and given the proper interface on `AbstractButton` and `AbstractWindow` it is completely agnostic of the concrete classes we have. But what if there are specialties for message windows?

If we implement that algorithm in the `FancyWidgetFactory` we do not win much, because `createButton` still returns a `unique_ptr<AbstractButton>`. We know that it is in fact a `FancyButton`, but we can not use that unless we apply some ugly downcasts.

std::unique_ptr<AbstractWindow> FancyWidgetFactory::createMessageWindow(std::string const& text) final override {
  auto theWindow = createWindow();
  theWindow->addText(text);
 
  auto theButton = createButton(); //unique_ptr<AbstractButton>
  static_cast<FancyButton*>(theButton.get())->doFancyStuff(); //EWW!
  theButton->setText("OK");
  theWindow->add(std::move(theButton));
  return theWindow;  
}

Covariance with raw pointers

In the olden days where open fire was considered more romantic than dangerous we would use raw pointers as return values from our factories. The callers would have to deal with ownership management, failing and burning down the house on a regular basis.

In those days, covariant return values were easy: A virtual function that returns a (raw) pointer can be overridden by a function that returns a pointer to a more derived class:

AbstractButton* OldschoolAbstractWidgetFactory::createButton();
FancyButton* OldschoolFancyWidgetFactory::createButton();

Since a `FancyButton` is an `AbstractButton`, this makes perfect sense, and the compiler knows that. With smart pointers it’s not so easy, since for the compiler they are only templates instantiated with two classes that happen to be related.

That relation does not transfer to the template instantiations, since it usually makes no sense. A `std::vector<Base>` is not related to a `std::vector<Derived>` as a `Base*` is related to a `Derived*`.

Achieving covariance with smart pointers

So now we know the problem. How do we solve it with the means the language allows us? Let’s analyze the situation:

  • We want `createButton` on an `AbstractWidgetFactory` to return something that holds a button. Which concrete button that will be depends on the concrete factory.
  • We want `createButton` on an `FancyWidgetFactory` to return something that holds a `FancyButton`, so we don’t need to cast.
  • We want to have smart pointers, but those are not considered covariant by the language.

The latter leads us to a simple conclusion: If we want the first two points to be true, `createButton` simply can not be `virtual`. The solution is, as it is so often, another layer of indirection. We can just give the factory classes a nonvirtual interface and let the virtual call take place in another function:

struct AbstractWidgetFactory {
  std::unique_ptr<AbstractButton> createButton() const {
    return doCreateButton();
  }
  // ...
private:
  virtual std::unique_ptr<AbstractButton> doCreateButton() const = 0;
};

struct FancyWidgetFactory : AbstractWidgetFactory {
  std::unique_ptr<FancyButton> createButton() const {
    return std::make_unique<FancyButton>();  
  }
  // ...
private:
  virtual std::unique_ptr<AbstractButton> doCreateButton() const final override {
    return createButton();
  }
};

We can now write the creation of our fancy message window without any ugly casts:

  std::unique_ptr<AbstractWindow> createMessageWindow(std::string const& text) final override {
    auto theWindow = createWindow();
    theWindow->addText(text);
 
    auto theButton = createButton(); //unique_ptr<FancyButton>
    theButton->doFancyStuff();       //no more casts
    theButton->setText("OK");
    theWindow->add(std::move(theButton));
    return theWindow;  
  }  

All this just works because `std::unique_ptr` to derived classes can always be converted into std::unique_ptr to their base class. Since this also applies to `std::shared_ptr` the same pattern can be used to achieve covariance with those.

But there’s a problem

As was discussed by rhalbersma in the comments, having the nonvirtual `createButton` method redefined in the derived class can lead to several issues. The most important one is that the behavior can be surprising to users, which never is a good thing.

The straight forward solution is to rename the method in the derived class, e.g. `createFancyButton`. That way the overall functionality remains, albeit it is more explicit and less surprising. This may not be the “true form” of covariance any more, but that are the kind of compromises we have to make.

You can find the full code on my GitHub repository.

Conclusion

If you really need covariance with smart pointers, it is manageable, although you have to add that extra layer of indirection. There should be better alternatives though, since C++ is not (only) an object oriented language.

There obviously is no perfect solution to the problem, but I hope I could show a possible approach to problems like this: If there is no single functionality that provides what we need, we can try to add another layer of indirection and combine the layers to produce the desired result.

Thanks to Jason Turner and Joshua Ogunyinka for bringing this topic up on twitter recently:

Facebooktwittergoogle_plusredditlinkedinFacebooktwittergoogle_plusredditlinkedinby feather

14 Comments


  1. This issue is more general than factory functions. The covariance requirement is for a transform F, such that if S < T, then F (S) F(T).

    This problem litters C++ because the type system is badly constructed. Here is another example: const. Const doesn’t propagate through templates.

    The heart of this problem is that nominal typing is specifically designed to require manual specification of all operations. If you want something done automatically you have to use structural typing with a carefully crafted minimal set of combinators.

    So what can you do? If you want covariant smart pointers then, you just have to implement the conversions. That is the whole intent of nominal typing. And you have to apply them as well. Can you do it? Well actually its trivial, for some “smart” pointer type P with a method get()->T* and a method set(T*) you can obviously just grab the pointer, do the cast, and set pointer in the result object. Templates can’t be type checked so you can write the code. If you use it where the conversion S* -> T* doesn’t exist your code will break on instantiation. So all you need now is to make that a constructor and you’re done.

    Reply
  2. Cristian

    Typo here: AnstractWidgetFactory

    Reply
  3. Nikita Kudyk

    I actually doubt that a Factory should return a smart pointer. Smart pointers represent the RAII idiom and one should not make every pointer smart. Smart pointers are the source of resource management. Factory itself obviously does not hold that resource so it should not take responsibility for resource management. Moreover, callers may have their own resource management schemes, so wrapping the pointer just to unwrap it the next step is a bit strange.

    Reply
    1. Arne Mertz

      The factory does not hold the resource, but it creates it. Therefore it should not return raw pointers but smart pointers. `unique_ptr` does not have overhead compared to owning raw pointers, so it is a good default. Any good custom resource management has built in ways to take over ownership from `unique_ptr` so this should not be an issue either.

      Reply
  4. emel

    Minor correction: in “Achieving covariance with smart pointers” you need the factory on this line:

    auto theButton = createButton(); //unique_ptr

    Reply
  5. alfC

    I never understood this problem very well, or the language design about this particular corner or the real uses.

    But isn’t it is telling us that factories should still return raw pointers and it is your responsibility to use the `unique_ptr` to get capture its value? (the real type information is still inside the `unique_ptr`)

    As an orthogonal justification, why should the factory know that you want a `unique_ptr` and not a `shared_ptr`? Perhaps raw pointers have its uses still, for example as return values.

    Reply
    1. Arne Mertz

      Having raw pointers returned from the factory is a design that may lead to exception safety issues. Think for example `someFunc(factory.createButton(), factory.createWindow())`. I discussed returning smart pointers from factory functions in another post here.

      Reply
  6. rhalbersma

    Sorry for an unfortunate error in my previous comment, please replace all occurrences of `createWindow` to `createButton` (which is the one you redefine). The argument itself remains the same.

    Another argument from Effective C++, Item 36: *”Never redefine an inherited non-virtual function”*. You get different behavior if you (or your users) accidently create another a reference/pointer of type`AbstractWidgetFactory` to your `theWidgetFactory` (because `createButton` is statically bound).

    Reply
    1. Arne Mertz

      Hi rhalbersma,
      thank you for pointing out those issues. Some I would argue against, but the most compelling argument is the surprise for the user. I’ll add an update to the post.

      Reply

  7. Thanks for sharing this useful technique.
    It is a nice alternative to using a template function, that would be specialized for fancy widgets.

    Reply
  8. rhalbersma

    Nice blog post! One nitpick: FancyWidgetFactory::createButton hides the name of AbstractWidgetFactory::createButton. It’s probably better to rename it to FancyWidgetFactory::createFancyButton, and also to make it a private helper function.

    Reply
    1. Arne Mertz

      Hi, and thanks for your comment.
      The whole point of `createButton` in the derived class is to override the behavior of the one in the base class by hiding its name. That way you mimic true covariant functions by having a function with the same name returning a more derived pointer.

      Reply
      1. rhalbersma

        I understand the mechanism but I think it is fragile and underhanded.

        It is fragile because redefining names from a base class in a derived class hides all names from the base. E.g. if `AbstractWidgetFactory` would add another overload `createWindow(int width, int height)`, then this would be hidden in `FancyWidgetFactory`. (I suspect that is why you have a `createMessageWindow(const char*)` instead of an overload `createWindow(const char*)`)

        To remedy this, you could write a `using AbstractWidgetFactory::createWindow` inside `FancyWidgetFactory`. If you want to guard your `FancyWidgetFactory` against future overload extensions in `AbstractWidgetFactory`, then you must write these `using` statements already today.

        The mechanism is also underhanded because you violate the is-a relationship between the base and derived factory class. E.g. if `AbstractWidgetFactory::createWindow` uses the NVI to do logging before calling `do_createWindow`, then you miss that in the derived class.

        I can’t think of an easy way to remedy the NVI, except to manually add logging in your redefined `createWindow`. This makes your derived factory vulnerable to future updates of the abstract factory (the logging mechanism might even be a private implementation detail of the base class, and you would have to guess about its exact reimplementation, especially if you don’t have the source of the base factory).

        It’s a pity that covariant return types don’t extend to smart pointers, but the combination of using public inheritance for your factory and redefining member functions is surprising and dangerous to users. IMO, they would be better served by having to explicitly call `FancyWidgetFactory::createFancyWindow` instead if they insist on receiving a `unique_ptr< FancyWindow>`.

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *