Even very young children know that sometimes, as you move, nearer objects will seem to move relative to more distant objects. Children wouldn't know the name of this phenomenon; it is called parallax. Scientists in the time of Copernicus and Galileo also knew about parallax. This knowledge led to the most serious and valid challenge to the theory that the earth moved; stellar parallax. They argued that if the earth moved around the sun as Copernicus and Galileo suggested, then there must be some evidence of parallax effects. A moving earth would mean that the earth would be on opposite sides of the sun six months apart. And if the earth was on opposite sides of the sun six months apart there should be some visible change in the relative position of the stars during the year (nearer stars should seem to move relative to more distant stars). Yet the relative positions of nearer and farther stars seemed to remain the same.
No one in Galileo's time or for almost 200 years after his death was able to demonstrate this necessary effect of earth's motion around the sun. Stellar parallax was finally observed in 1838 by Friedrich Bessel, a German scientist. But it is not Bessel that is credited with finally proving that the earth moved around the earth. In 1725, James Bradley, while searching for the elusive stellar parallax, detected motion of the stars over the course of the year which did not fit the pattern of stellar parallax. He had discovered stellar aberration, which is also related to the motion of the earth. Regardless, proof of the earth's motion was not available in the seventeenth century and those arguing for it's motion had no answer for why stellar parallax could not be observed. If there is a necessary consequence of a theory and that consequence cannot be observed that is a huge problem for any scientific proposition. It is enough to keep a hypothesis from being accepted as a proven theory, regardless of the number of positive arguments in its favor. This applies to science today and it also applied in the seventeeth century.
While this criticism of a sun-centered solar-system may have been scientifically valid at the time it does illustrate a kink in the scientific approach which exists even today. These scientists were correct that if there was a moving earth then there must be stellar parallax. They were also correct that this parallax could not be observed. However, their rejection of the hypothesis was based on more than this; it was also based on the assumption that the stars were millions of miles away. These stars were billions of miles away. Given the instrumentation of the day, one could expect to detect stellar parallax if the stars were millions of miles away but not if they were billions of miles away.
The difficulty of eliminating assumptions in testing theories exists as much today as it did four centuries ago. It even has a name; the Duhem-Quine thesis. It was originally proposed by the great philosopher of science, Pierre Duhem. Simply put, this thesis states that in practice it is difficult to ever test a theory independently of other theories or assumptions. This means that when an experiment 'proves' a theory false it is really just proving the collection of theories and assumptions false, not necessarily the theory itself. This has implications for important themes in the modern philosophy of science, especially falsification. Interestingly, Pierre Duhem is the same person who as a historian discovered that there were important scientific advances in the middle ages, largely attributable to church scientists (see Duhem and the DaVinci Code). Duhem was also one of the most prominent victims of academic censorship in the twentieth century largely due to his historical work.