Narrow the field with what you already know. Jason Prothero suggests starting from your team. Which technology stack does your IT department work with, for example Microsoft or Linux? Who'll help maintain and develop the site, and which technologies do they know?
Shortlist three or four options, ideally of different types, so you see real alternatives. Then test them yourself. A sales demo shows the best path through the product; your editors will meet all the other paths.
Give your editors real tasks, with your own content:
Create a new page with text, images and a form, and publish it.
Reuse a content block on several pages, then change it once.
Translate a page into a second language.
Schedule a page and send it for approval.
Roll back to an earlier version of a page.
Watch whether the editing experience suits your team. Jason noted that experienced editors can handle a lot of design freedom, while others get overwhelmed by too many options and work better with a more controlled setup.
Ask to see the backoffice of real, live sites, as well as the demo environment. Emiliano Bottaioli recommends asking agencies to show how they implemented earlier projects. Few buyers ask to see the backoffice, even though editors will live in it every day.
If you write an RFP, describe scenarios and keep it focused.
"Our marketing team needs to launch a campaign page in three languages without developer help" tells you far more than "Multilingual: yes/no".
Emiliano's warning: when an RFP lists everything, agencies have to price everything, and you pay for capabilities you may never use.
Nik's advice: aim for a handful of quality responses rather than fifteen, and talk to the bidders instead of judging on price alone.