To complete the sentence you incompletely quoted: "but in deference to the opinions and wishes of the other slaveholding States, she forbore at that time to exercise this right."
What are those "frequent violations of the Constitution"?
> In the present case, that fact is established with certainty. We assert that fourteen of the States have deliberately refused, for years past, to fulfill their constitutional obligations, and we refer to their own Statutes for the proof.
> The Constitution of the United States, in its fourth Article, provides as follows: "No person held to service or labor in one State, under the laws thereof, escaping into another, shall, in consequence of any law or regulation therein, be discharged from such service or labor, but shall be delivered up, on claim of the party to whom such service or labor may be due."
> This stipulation was so material to the compact, that without it that compact would not have been made. The greater number of the contracting parties held slaves, and they had previously evinced their estimate of the value of such a stipulation by making it a condition in the Ordinance for the government of the territory ceded by Virginia, which now composes the States north of the Ohio River.
> The same article of the Constitution stipulates also for rendition by the several States of fugitives from justice from the other States.
> The General Government, as the common agent, passed laws to carry into effect these stipulations of the States. For many years these laws were executed. But an increasing hostility on the part of the non-slaveholding States to the institution of slavery, has led to a disregard of their obligations, and the laws of the General Government have ceased to effect the objects of the Constitution.
It's all about the injustice that they are not able to keep slaves like they want to.