Whenever a meeting ended, he would lean back in his chair and mutter, as he used to say, 'Well, that's that.'

as he used to say 是以 as 引出的叙事插入分句，为叙述添加背景——这是说话者过去常挂在嘴边的口头禅。used to 为半情态动词结构，标记过去的惯常行为。