I have no problem changing rows and columns to 5 and having it modfiy my layout.
Are you using the default theme or a different theme?
Did you go to Edit Album -> Layout -> modify rows and columns, save? Keep in mind I believe unless you click global it will only do the album your in/selected. So if your top album only has 4 items you won't see a change necessarily. But if your expecting it to change the sub-albums because they have more than 5 albums then I believe you need to select global or just edit that particular album.