I've got a sharp eye, but at 300k for Brick, vs 92k for Google Fonts, I've got to go with Google Fonts here, especially when I'll usually have a given Google Font cached anyways, so it's often 0. But I'm glad to be given the option, regardless, and the opportunity to compare.
And as for character sets, Google Fonts lets you pick the character sets you want, too. Does Brick provide any characters which Google doesn't allow you to add?