[Video] Tutorial how to Upgrade Smart Contracts on Chain

It's not easy to get Upgradeable Smart Contracts right. You should be able to fix bugs or enhance features, obviously. But not at the cost of loosing immutability. It's this element of trust that nobody - not even a trusted entity - can change anything. Thinking of censorship, government bans, and so on...

In this video I am going to talk about different approaches how smart contracts can be upgraded directly on chain and compare the solutions.

Link to YouTube

The topic of upgradeable smart contracts is nothing particularly new to the Ethereum world. The problem is, no matter the way of the implementation, there is always some advantage and some drawback. In general there is a good reason for and against being able to upgrade contracts.

The most important part of immutability: It makes sure that nobody can make changes afterwards. This element of trust is what makes Ethereum and Smart Contracts so incredibly powerful. It's like burning the logic into the blockchain. You release your code - and bam - it's there. On chain. Nobody can change it or take it down.

But on the other hand, recent hacks were all based on very simple programming errors. Those bugs could be fixed very easily, if it was possible to upgrade these contracts in one way or another.

The Intuitive Approach

Let's have a look at the first very intuitive approach: release a new contract and move all the data over.

For example this very simplistic contract that stores a public unsigned integer "myVar":

pragma solidity ^0.4.18;

contract SomeContract {
    
    uint public myVar;
    
    function SomeContract(uint _myVar) public {
        myVar = _myVar;
    }
    
    function getMyVar() public view returns(uint) {
        return myVar;
    }
    
}

I will walk step by step through this smart contract. There is one public unsigned integer "myVar". In the constructor of the conract we set the variable and there is one function "getMyVar()" which returns the variable.

What if we want to change the "getMyVar()" Function so that it returns 5*myVar? We could just release a new version of "SomeContract", we could read the variable "myVar" from the old contract first and set it in the new version right. But there are some problems with that.

  1. What if we release a new contract but make a mistake during setting the variables
  2. With only one single variable it's all pretty convenient, but what if there are hundreds or thousands?

Easy to see that this doesn't scale well and it's not very trustworthy.

Problem With Imports and Libraries

Now another intuitive approach would be to just have the logic separated and imported into a bigger "main" contract and use if from there. Let me give you an example:

pragma solidity ^0.4.18;

library UserLib {
    
    struct DataStorage {
        mapping(address => bool) addressSet;
    }
    
    function userNameExists(address self, DataStorage storage _dataStorage) public view returns (bool) {
        return _dataStorage.addressSet[self];
    }
    
    function setUsername(address self, DataStorage storage _dataStorage) public {
        _dataStorage.addressSet[self] = true;
    }
    
}

contract UserContract {
    UserLib.DataStorage dataStorage;
    
    using UserLib for address;
    
    function isMyUserNameRegistered() public view returns(bool) {
        return msg.sender.userNameExists(dataStorage);
    }
    
    function registerMe() public {
        msg.sender.setUsername(dataStorage);
    }
}

If we try to compile the UserContract, we end up with the following instruction for web3:

var browser_userlib_sol_usercontractContract = web3.eth.contract([{"constant":false,"inputs":[],"name":"registerMe","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"isMyUserNameRegistered","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"view","type":"function"}]);
var browser_userlib_sol_usercontract = browser_userlib_sol_usercontractContract.new(
   {
     from: web3.eth.accounts[0], 
     data: '0x6060604052341561000f57600080fd5b61026d8061001e6000396000f30060606040526004361061004c576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff1680639a198d6114610051578063c368ec0614610066575b600080fd5b341561005c57600080fd5b610064610093565b005b341561007157600080fd5b610079610160565b604051808215151515815260200191505060405180910390f35b3373ffffffffffffffffffffffffffffffffffffffff1673__browser/UserLib.sol:UserLib___________63dec1380d909160006040518363ffffffff167c0100000000000000000000000000000000000000000000000000000000028152600401808373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020018281526020019250505060006040518083038186803b151561014a57600080fd5b6102c65a03f4151561015b57600080fd5b505050565b60003373ffffffffffffffffffffffffffffffffffffffff1673__browser/UserLib.sol:UserLib___________63571485129091600080604051602001526040518363ffffffff167c0100000000000000000000000000000000000000000000000000000000028152600401808373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020018281526020019250505060206040518083038186803b151561022157600080fd5b6102c65a03f4151561023257600080fd5b505050604051805190509050905600a165627a7a72305820dafb58e27079d9de9eab1fce2248784de98b97c870c0d793c617369a721ebaec0029', 
     gas: '4700000'
   }, function (e, contract){
    console.log(e, contract);
    if (typeof contract.address !== 'undefined') {
         console.log('Contract mined! address: ' + contract.address + ' transactionHash: ' + contract.transactionHash);
    }
 })

You can see the placeholders for the library inside the bytecode. So, the linker would go in there and put in the Libraries address at the exact position of the "browser/UserLib.sol:UserLib_________".

Now intuitively we want to upgrade the library to a newer version. If we do that we'd end up with a new address for a new library. Now we can't update the UseContract Contract, because the address of the old library is fixed in the bytecode which we are deploying.

Some Solutions

Before we dive into our own solution, let's have a look what's out there already.

1. Monax

Let's start with Monax.io Solidity Tutorials. You can find them here:

https://monax.io/docs/tutorials/solidity/solidity_1_the_five_types_model/

They are a great intro into upgradeable Contracts based on a larger series of in-depth but older Solidity Tutorials. Those tutorals are very well written for larger scale architectures. They resemble what software architecture is all about. Making it abstract where necessary, upgradeable, reusable, and so on.

2. Proxy Contracts from Zeppelin

Zeppelin Soltions uses a User-facing dispatcher contract that talks to an underlying library with assembly. It tricks the Transaction-Initiator into talking to an actual fallback function and then delegating the call to a library. Drawback: this dispatcher contract must know the memory size of the return value. It also comes with a separate DispatcherStorage because delegatecall to a library can't work when libraries can't actually save any values...

You can find their solution here in their blog post https://blog.zeppelin.solutions/proxy-libraries-in-solidity-79fbe4b970fd

Or on their Repository https://github.com/maraoz/solidity-proxy/

3. Ether-Router

EtherRouter is another method, quite similar to the Solidity Proxy solution from Zeppelin. The one who initiates the transaction is actually talking to a fallback function from the Ether-Router. This Contract determines then the address of the actual contract and delegates the call there via assembly. The whole implementation can be found here on their GitHub-Page:

https://github.com/ownage-ltd/ether-router

4. Blog from Elena Dimitrova

Then there is this blog post from Elena Dimitrova, which describes the best technique in my opinion. Instead of trying to "outsource" the logic, you put the storage somewhere else.

The idea is rather simple: There is one smart contract called "EternalStorage" which is a pure storage-contract without the logic. And another contract which can access the EternalStorage contract. You can have as many contracts accessing the EternalStorage contract as you want, and hence update the logic part without sacrificing the storage.

Here is the link:

https://blog.colony.io/writing-upgradeable-contracts-in-solidity-6743f0eecc88

5. Follow-up posts

Obviously, the initial Blog post caught a lot of attention. And it didn't take long that some follow-up posts were published based on the initial EternalStorage idea.

The EthernalStorage is also not flawless. It introduces the first attempt before the Metropolis HardFork. Before Metropolis it wasn't possible to interact with dynamic byte size arrays such as strings, so the follow posts fixed this and some other parts as well.

In particular I want to highlight the latest Article I found: https://medium.com/rocket-pool/upgradable-solidity-contract-design-54789205276d

Rocket Pool is laying out how they use the EternalStorage, introduced in Elena's post, to power their infrastructure of upgradeable contracts. They posted the improved version of the EternalStorage, which can now save almost anything you like.

But what's behind? Let's have a look at our very very simple example for saving Users from before.

Outsource the Storage

We take again our UserContract but this time we are not linking a library, we are using a storage contract. The storage contract is obviously not as sophisticated, nor powerful as the EternalStorage contract, but it will show you what is meant by upgrading smart contracts.

First, let me show you the two contracts before explaining step by step what everything means:

pragma solidity ^0.4.18;

contract UserStorage {
    
    mapping(address => bool) addressSet;

    function getAddressSet(address _address) public view returns(bool) {
        return addressSet[_address];
    }
    
    function setAddressSet(address _address, bool _bool) public {
        addressSet[_address] = _bool;
    }
    
}

contract UserContract {
    
    UserStorage userStorage;
    
    function UserContract(address _userStorageAddress) public {
        userStorage = UserStorage(_userStorageAddress);
    }
    
    function isMyUserNameRegistered() public view returns(bool) {
        return userStorage.getAddressSet(msg.sender);
    }
    
    function registerMe() public {
        userStorage.setAddressSet(msg.sender, true);
    }
}

We have two contracts here, one UserStorage and one UserContract. The UserStorage has only logic to store data, it doesn't do anything else. The UserContract has the logic that interacts with the UserStorage. Well, there isn't much logic in there yet, but if, let's say, we were to update the logic, we can update the UserContract without loosing the data, or the need to migrate the data.

The UserStorage has one mapping that stores a boolean to an address. You can either write or read. Now, obviously, everyone can do this, so we might want to change the overall behavior so only our UserContract can read/write:

We could add a new mapping, that stores who has access, add a modifier (I call it platform) so that only "platform public" access is allowed (everything within the authorized platform) and add getters and setters to change who is allowed or not:

pragma solidity ^0.4.18;

contract UserStorage {
    
    mapping(address => bool) addressSet;
    
    mapping(address => bool) accessAllowed;
    
    function UserStorage() public {
        accessAllowed[msg.sender] = true;
    }
    
    modifier platform() {
        require(accessAllowed[msg.sender] == true);
        _;
    }
    
    function allowAccess(address _address) platform public {
        accessAllowed[_address] = true;
    }
    
    function denyAccess(address _address) platform public {
        accessAllowed[_address] = false;
    }
    

    function getAddressSet(address _address) public view returns(bool) {
        return addressSet[_address];
    }
    
    function setAddressSet(address _address, bool _bool) platform public {
        addressSet[_address] = _bool;
    }
    
}

Now, in order to run this, we have to

  1. Deploy the UserStorage contract
  2. Copy the address from UserStorage and use it in the constructor of UserContract
  3. Add the UserContract address to the "accessAllowed" mapping of the UserStorage

How to implement this in Truffle

Having the general structure in Remix, we want to add it into our build process with truffle. Essentially we want to achieve four things:

  1. Deploy the UserStorage only, and only if it wasn't deployed already (don't deploy it a second time)
  2. Deploy the UserContract
  3. Link them together
  4. Update the Website accordingly to use the new UserContract if changes occurred.

For this I will take a different example. I'll use the MetaCoin example which comes with truffle-webpack.

First, let's install truffle if you haven't.

Truffle installation and initalization

Truffle comes as a node package via the Node Package Manager (npm). You need to install Node.js which includes the Node Package Manager. Then you can install truffle via the command line.

On windows I'd suggest you use the PowerShell, on MacOS and Linux you can use any terminal.

You can type in npm install -g truffle and it should download and install truffle, which is at the time of writing version 4.0.1:

install_truffle

Now you are ready to use truffle globally. In any empty directory you can use truffle unbox webpack and truffle will download the truffle-webpack repository and install all dependencies.

truffle_unbox_webpack

Edit the MetaCoin Contract

In this section we are going to edit the standard MetaCoin example that comes with truffle-webpack.

On a sidenote, I recently discovered that Atom has not only syntax-highlighting for Solidity, but also Autocompletion and linting. Very handy, in case you are wondering what editor I'm using.

Now, here is the general structure opened in my editor:

atom_truffle_webpack

We have a MetaCoin contract, which is the main part of the whole webpack example. We also have a website in /app, which is the web-part with web3.js.

Now, let's identify the storage elements of the MetaCoin example. There is luckily only one single element, which is mapping(address => uint) balances. We will get this out of the MetaCoin contract.

Let's create a new contract called "MetaCoinStorage". Mostly it's copying over the UserStorage contract with the modifiers for checking if the access is allowed. But it is also changing the addressSet mapping to the balances mapping. Here is the final contract:

pragma solidity ^0.4.18;

contract MetaCoinStorage {

    mapping(address => uint) balances;

    mapping(address => bool) accessAllowed;

    function MetaCoinStorage() public {
        accessAllowed[msg.sender] = true;
        balances[tx.origin] = 10000;
    }

    modifier platform() {
        require(accessAllowed[msg.sender] == true);
        _;
    }

    function allowAccess(address _address) platform public {
        accessAllowed[_address] = true;
    }

    function denyAccess(address _address) platform public {
        accessAllowed[_address] = false;
    }


    function getBalance(address _address) public view returns(uint) {
        return balances[_address];
    }

    function setBalance(address _address, uint _balance) platform public {
        balances[_address] = _balance;
    }

}

And now let's change the MetaCoin contract that it uses the MetaCoinStorage instead of directly accessing local variables:

pragma solidity ^0.4.2;

import "./ConvertLib.sol";
import "./MetaCoinStorage.sol";

// This is just a simple example of a coin-like contract.
// It is not standards compatible and cannot be expected to talk to other
// coin/token contracts. If you want to create a standards-compliant
// token, see: https://github.com/ConsenSys/Tokens. Cheers!

contract MetaCoin {
	MetaCoinStorage metaCoinStorage;

	event Transfer(address indexed _from, address indexed _to, uint256 _value);

	function MetaCoin(address metaCoinStorageAddress) {
		metaCoinStorage = MetaCoinStorage(metaCoinStorageAddress);
	}

	function sendCoin(address receiver, uint amount) returns(bool sufficient) {
		if (metaCoinStorage.getBalance(msg.sender) < amount) return false;
		metaCoinStorage.setBalance(msg.sender, metaCoinStorage.getBalance(msg.sender) - amount);
		metaCoinStorage.setBalance(receiver, metaCoinStorage.getBalance(receiver) + amount);
		Transfer(msg.sender, receiver, amount);
		return true;
	}

	function getBalanceInEth(address addr) returns(uint){
		return ConvertLib.convert(getBalance(addr),2);
	}

	function getBalance(address addr) returns(uint) {
		return metaCoinStorage.getBalance(addr)+10;
	}
}

Let's see what changed. First, we need to import the MetaCoinStorage contract. Then we need to assign the address of the MetaCoinStorage contract in the constructor. In addition to that, we need to read/write to the MetaCoinStorage contract every time we make a change to the balance, instead of directly to the MetaCoin "balances" mapping. So, all functions use metaCoinStorage.getBalance(...) instead of balances[...].

Now comes the tricky part. Let's say we found a bug in our MetaCoin Contract. You might have noticed that I accidentally left a bug in the function "getBalance(...)" of the MetaCoin contract. Instead of outputting the balance, it will output the balance + 10. Here is the part, at the end it says "+10":

	function getBalance(address addr) returns(uint) {
		return metaCoinStorage.getBalance(addr)+10;
	}

Before we go ahead and fix the bug, let's see how we can deploy these contracts using the migrations of truffle.

I will expand the migrations and add a new one, because

  1. first, we have to deploy the MetaCoinStorage then
  2. we have to deploy the MetaCoin contract
  3. last we have to allow the new MetaCoin contract to access the MetaCoinStorage contract

Here are my 3 migrations which are under the /migrations folder in Atom:

1_initial_migration.js stays exactly the same. 2_deploy_Storage.js is different, as it only deploys the MetaCoinStorage contract:

var MetaCoinStorage = artifacts.require("./MetaCoinStorage.sol");

module.exports = function(deployer) {
  deployer.deploy(MetaCoinStorage);
};

And the last one 3_deploy_MetaCoin.js, which deploys the MetaCoin contract and then allows the newly deployed address access to the MetaCoinStorage contract:

var ConvertLib = artifacts.require("./ConvertLib.sol");
var MetaCoin = artifacts.require("./MetaCoin.sol");
var MetaCoinStorage = artifacts.require("./MetaCoinStorage.sol");

module.exports = function(deployer) {
  deployer.deploy(ConvertLib);
  deployer.link(ConvertLib, MetaCoin);
  deployer.deploy(MetaCoin, MetaCoinStorage.address).
  then(() => {
    MetaCoinStorage.deployed().then(inst => {
      return inst.allowAccess(MetaCoin.address);
    });
  });
};

Now, let's start our development console and make sure we get everything right.

truffle develop starts the development console:

truffle_develop

then we do a truffle migrate:

truffle_migrate_1

It will pop out some warnings, because the MetaCoin examples inside the truffle-webpack project are without a visibility identifier (public). For now we can safely ignore this.

Now let's check directly on the command line how many coins we have in our first account. Let's run a command:

MetaCoin.deployed().then(inst => { return inst.getBalance.call(web3.eth.accounts[0]);}).then(bigNum => { return bigNum.toNumber();});

It should output that we have 10010:

truffle_command

Obviously we should only have 10000 inside our contract. Still, let's assume we didn't find this bug immediately and let's send some coins around. Let's send 5000 coins from account-0 to account-1:

Again, directly on the truffle developer console, input this:

MetaCoin.deployed().then(inst => { return inst.sendCoin(web3.eth.accounts[1], 5000);});

It should output the transaction-result object:

truffle_send_coin

Let's check the balance again to make sure we just have 5000 coins left:

truffle_check_balance

Bummer! We have 5010 instead of 5000 coins. Let's fix the bug and re-deploy our contract. Our MetaCoin contract with the bugfix looks like this:

pragma solidity ^0.4.2;

import "./ConvertLib.sol";
import "./MetaCoinStorage.sol";

// This is just a simple example of a coin-like contract.
// It is not standards compatible and cannot be expected to talk to other
// coin/token contracts. If you want to create a standards-compliant
// token, see: https://github.com/ConsenSys/Tokens. Cheers!

contract MetaCoin {
	MetaCoinStorage metaCoinStorage;

	event Transfer(address indexed _from, address indexed _to, uint256 _value);

	function MetaCoin(address metaCoinStorageAddress) {
		metaCoinStorage = MetaCoinStorage(metaCoinStorageAddress);
	}

	function sendCoin(address receiver, uint amount) returns(bool sufficient) {
		if (metaCoinStorage.getBalance(msg.sender) < amount) return false;
		metaCoinStorage.setBalance(msg.sender, metaCoinStorage.getBalance(msg.sender) - amount);
		metaCoinStorage.setBalance(receiver, metaCoinStorage.getBalance(receiver) + amount);
		Transfer(msg.sender, receiver, amount);
		return true;
	}

	function getBalanceInEth(address addr) returns(uint){
		return ConvertLib.convert(getBalance(addr),2);
	}

	function getBalance(address addr) returns(uint) {
		return metaCoinStorage.getBalance(addr);
	}
}

But how can we re-deploy the contract without changing the storage-layer? Truffle has us covered here, we can just run a single migration:

truffle migrate -f 3 will run migration #3:

truffle_migrate_3

As you can see, it compiles everything, but migrates only 3_deploy_MetaCoin.js. Now let's check our balance again. It should be 5000 now:

truffle_check_balance_3

This way, by having the storage in one contract and the logic in another one, it's easy to upgrade the logic part.

How this compares to the EternalStorage or RocketStorage

Now, let's have a quick look at the "EternalStorage" contract from Elena, or from RocketPool. I recommend to go through the one from RocketPool here, because it's more up2date after the Metropolis upgrade.

The contract is made for storing arbitrary types of data, which might be something you want or not. Depending on your use-case it can be good to have a Storage-Contract which you can extend with new data or variables based on a key-value system. On the other hand, sometimes it's good to keep it as simple as possible. That's for the software architect to decide, in my opinion there is no simple answer to it.

One thing that sould be clear: A storage contract should have as little logic as possible. Additionally it must be made sure that every single part of the storage is tested. Once the storage is deployed, it can't be changed, so be careful.

Benefits and Drawbacks

After several hundred million USD being stolen or lost because of simple programming errors, it's clear that sometimes it makes sense to be able to introduce updates. Updates that are for fixed or code enhancements.

One of the biggest drawbacks is exactly the fact that you can overwrite the logic. This means, everybody has to trust you that you're the good guy. There is no solution to that to my best knowledge. One thing that might makes sense would be to have multi-sig-upgrades, where the "OK" from multiple people is necessary before a new contract is deployed and can access the storage.

Final Conclusion

Architecture-wise, Solidity and the Ethereum Ecosystem finally reaches a point where updates are possible. The implementation is not quite there yet to make use of all the features the underlying Ethereum "platform". Also there is no one-fits-all solution yet available, but there are several approaches which you might find beneficial when deploying your next distributed application.

Like or dislike this article? Let me know in the comments and subscribe.