Understood (see above). The issue here isn't so much that I want to add custom colors. Rather, when I'm creating a diagram with just a few colors (the normal case), I will constantly setting most of the objects to just those colors. However this requires some precision pointing to be sure I grab the correct red or green or blue or whatever. I will often notice later on that I accidentally hit the adjacent color entry on one object. The MS Office UI element with 'recently used colors' showing at the bottom of the chooser (and even more: the default operation if you click the 'set color' button rather than the down-arrow that brings up the chooser) is a real convenience that I use constantly. It avoids the need for a precise mouse operation so I can work more quickly.
But again, if you're using an inherited color picker this is basically moot. Either they'd have to add this feature to the platform, or you'd have to decide this was important enough to build your own (unlikely until you have a team of a hundred engineers working on this tool, heh heh).