York H. Dobyns says (Oct 30, '92): +quot;I'm no fan of the Mars effect: the evidence is qu
---------------------- Message ---------------------------------
York H. Dobyns says (Oct 30, '92): "I'm no fan of the Mars
effect: the evidence is questionable due to methodological
flaws." Nevertheless he then dwells meticulously on consequences
that would arise if the Mars effect were shown to be real.
Apparently, the evidence is not questionable enough to remain
aloof from it.
My business today is to question the evidence for
"methodological-flaws" arguments abounding in discussions here
about the Mars effect. Most of these arguments have been brought
forward by Jan Nienhuys. Newsgroup members seem to readily take
his views as their own. What I would like to suggest is to read
his messages more skeptically (should not be difficult in this
circle). I am going to tell you why.
---------------------------- 1 -------------------------------------
J.N. Nienhuys
28 Oct 1992
> Later more "effects" were found with different professions.
> But (to me) the main point is that Lasson already emphasized
> one should take "famous" professionals. It's quite natural
> that if the reality of this effect was suspected, they (both
> Gauquelins) would try to determine how famous or good the professional
> had to be. It is also conceivable that a psychologist receiving
> his statistics training in the early '50's would not be aware of
> artifacts resulting from cumulative biases introduced by this type
> of exploration.
News-readers, not being familiar with Gauquelin's procedure in
detail will take it for granted that Nienhuys justifiably
refers to some actual bias connected per se with collecting birth
data of famous people. They will find it plausible that artefacts
did accumulate due to poor statistical knowledge of the Gauquelins
in the Fifties. "It wasn't their fault (how considerate we are),
researchers at that time were not as sophisticated as we are today
(how excellent we are)". The verdict is done.
I feel obliged to defend Michel Gauquelin (he died
last year and thus cannot defend himself) one of the most
admirable figures (admirable regarding methodological
conscientiousness) in frontier science fields.
First, there is no bias at all associated with collecting birth
data with preference for eminent people. On the contrary,
collecting birth data without considering eminence must be
considered as severely biased. As soon as Gauquelin had reported
that planetary effects were stronger with famous than with
average athletes (his first observation) any subsequent study
testing the replicability of planetary effect was bound to
select more excellent individuals from professional samples.
Both, the American and the French skeptics did gravely
(CSICOP) or appreciably (CFEPP) violate the eminence
requirement. A study aiming at refuting some purported effect
must show its absence despite having established most
favorable conditions for the purported effect to occur.
CSICOP/CFEPP apparently did not set up such conditions. They
collected data in a way as if they feared the Mars effect might
emerge (see data below).
---------------------------- 2 -------------------------------------
J.N. Nienhuys
28 Oct 1992
> Given the rampant habit of reporting "significance" without
> model or hypothesis prior to the experiment (rampant at least
> in social science and medicine), there is absolutely nothing special
> or fraudulent about one psychologist not making the proper
> distinction between exploration and testing.
Jan Nienhuys here claims that the Gauquelins did not make proper
distinction between exploration and testing. The facts tell a
totally different story: M. Gauquelin published results of an
exploratory study in 1955 based on French data. Since then most
of his publications were of the hypothesis-testing kind. In 1960
he published his first hypothesis-testing study based on
Italian, German, Belgian, and Dutch data. The book gives answers
to the question: "Do previous French results replicate with
non-French data? The chapters provide reports on hypotheses,
subjects, methods of analysis, results, discussions as to
whether the hypotheses had or had not been supported (etc.).
On reading statements as those made by Nienhuys turning the
facts upside down and on having to witness their acclamation by
the majority of responding readers ("Congratulations, Jan!") I
just feel sad, and doubts arise as to whether justice and
fairness has any better chance to prosper in our science
community than elsewhere in this foul world.
---------------------------- 3 -------------------------------------
How eminent were athletes selected by the skeptics research
groups as compared to athletes selected by Gauquelin? Here we
exclude athletes common in both samples, i.e., we only consider
athletes listed either in the skeptics ("CSICOP-only",
"CFEPP-only") or in Gauquelin's sample ("Gauquelin-only"). The
eminence of an athlete is defined by the occurrence of his/her
name in 18 reference sources.
The CSICOP-only sample of U.S. athletes is compared with a
Gauquelin-only sample of U.S. athletes (which G. collected
later), the CFEPP-only sample of French athletes is compared
with the Gauquelin-only sample of French athletes (which G.
had collected earlier) (see Table 1).
Table 1: Numbers of American and French athletes
in skeptics and Gauquelin samples.
American: Kurtz et al. (CSICOP) N = 216
American: Gauquelin .......... N = 162
French: CFEPP ............... N = 398
French: Gauquelin ........... N = 130
Table 2 shows citation counts in percentages of respective Ns for
the skeptics' and Gauquelin's citation subsamples, as well as the
differences "Gauquelin - skeptics."
Table 2: Citation counts (percentages)
---------------------------------------------------------------
CSICOP Gauquelin Difference GAUQ-CSICOP
---------------------------------------------------------------
citations = 0 72.5 25.9 -46.6
citations = 1 24.9 22.8 - 2.1
citations = 2 2.6 29.1 26.5
citations = >2 0.9 22.2 22.2
---------------------------------------------------------------
100.0 100.0
---------------------------------------------------------------
---------------------------------------------------------------
CFEPP Gauquelin Difference GAUQ-CFEPP
---------------------------------------------------------------
citations = 0 71.9 36.9 -35.0
citations = 1 18.1 20.8 2.7
citations = 2 3.8 23.1 19.3
citations = 3 4.3 11.5 7.2
citations > 3 2.0 8.7 6.7
---------------------------------------------------------------
100.0 100.0
---------------------------------------------------------------
Results
1) The majority of CSICOP-only athletes is much less
eminent than Gauquelin-only athletes.
2) The majority of CFEPP-only athletes is much less eminent than
Gauquelin-only athletes.
Considering this difference alone the skeptics' studies cannot be
taken as appropriate for testing Gauquelin's claim of a Mars
effect. However, despite unfavorable preconditions a significant
overall Mars G% deviation was observed at least for the French
skeptics' sample. (This has been reported earlier).
---------------------------- 4 -------------------------------------
Another way of testing the presence of a Mars effect consists of
calculating G% for each eminence subsample separately and to
test for monotonic trend i.e., to test for the claim that G%
increases with athletic fame. Results from such trend test is
independent of the general G% level of the entire sample
(=citation subsamples pooled) and may be considered as an
alternative test of planet-birth relationships.
Table 3 shows G% values for the skeptics and the Gauquelin
samples, first U.S., then French athletes, entire samples,
broken down by citation frequencies.
Table 3: G% (key sector percentages) for the skeptics'
and Gauquelin athletes' samples.
----------------------------------------------
CSICOP Gauquelin
N=408 N=349
----------------------------------------------
citations = 0 18.1 21.4
citations = 1 21.5 21.3
citations > 1 25.9 30.2
----------------------------------------------------
CFEPP Gauquelin
N=1076 N=2040 (publ. and unpubl.)
----------------------------------------------------
citations = 0 22.8 26.1
citations = 1 28.6 25.1
citations = 2 24.6 27.6
citations = 3 27.9 26.4
citations > 3 28.5 31.1
----------------------------------------------------
Note: Less famous athletes (less citations) are much greater in
number than famous athletes. Therefore, for small total
samples of athletes famous subsample will be
sufficiently large only by pooling individuals from a
greater range of citation frequencies.
Results:
1) A monotonic increase of G% with citation counts is present in
the Gauquelin data, both samples.
2) A monotonic increase of G% with citation counts is present in
CSICP's data (Kurtz et al.)
2) A monotonic increase of G% with citation counts is present in
CFEPP's data (Benski)
---------------------------- 5 -------------------------------------
J.N. Nienhuys
29 Oct 1992
> There may be a file drawer effect. When Ertel asked Gauquelin
> in detail *all* his data, not only the ones he had published,
> he found that in the file drawer there were many athletes (unpublished)
> that did collectively *not* have a Mars Effect (namely around 22%
> born `under Mars'), and even did not conform to the average number
> of people (17.2%), but that were collectively born `under Mars'
> in a much smaller fraction. Their distribution over the sectors was
> a kind of mirror image of the distribution of the published athletes.
> Details:
> French published athletes: 1357 of which born `under Mars' :306
> French unpublished : 683 89
> Nonfrench published : 1531 322
> Nonfrench unpublished : 820 133
> Readers with calculators (and knowing just a smattering of statistics)
> are invited to draw their own conclusions.
> JWN
Jan Nienhuys here gives a partial summary of my published report
about Gauquelin's selection procedure. His summary is partial
for three reasons:
First, Nienhuys would like to make his readers believe that
discarding and not publishing cases must be regarded per se as
bias. I discussed this point in my paper from which he took the
information so he should know - and should have told his readers
- that not publishing cases in Gauquelin's study was not at all
bias per se. Gauquelin was not merely entitled, he even HAD to
exclude from his athletes sample individuals of lower eminence
rank after having found in 1955 that the Mars effect
increased with eminence. He was no less bound to throw mediocre
figures out of his sample than the skeptics in their
replications were bound to do just that (as discussed above).
Comparing unpublished with published Gauquelin atheletes
regarding c i t a t i o n c o u n t s (see Table 3) we do find
that the numbers of citations is considerably lower for
unpublished as compared to published athletes. Gauquelin thus
actually SUCCEEDED, by discarding cases, to improve the
general level of success in his sample.
Table 3.
Citation percentages for published and
unpublished Gauquelin athletes
citations Published Unpublished Difference
(N=2888) (N=1503)
-----------------------------------------------
0 46.1% 62.5% 16.4%
1 18.9% 29.4% 10.5%
2 16.0% 4.9% -11.1%
3 7.3% 2.9% -4.4%
4 3.3% 0.2% -3.1%
5 2.7% 0.0% -2.7%
6 1.1% 0.0% -1.1%
7 0.6% 0.0% -0.6%
8 0.1% 0.0% -0.1%
---------------------------------------
100.0% 100.0%
---------------------------------------
Nevertheless, Gauquelin should have done this discarding
(and not publishing) athletes with greater care. He should have
discarded low achievers prior to collecting their birthdates. Or
else, after having obtained their birth data, he should have
asked some naive assistant to check biographical information and
to discard low achievers by applying some reasonable criteria of
success. Gauquelin did such selections himself, obviously
overestimating his ability to base his judgment solely on the
athletes record of successes without considering, at the moment
of decision, his possibly knowing/remembering the respective
person's planetary positions. I searched for this inflating bias
and I found and published the result.
When Gauquelin heard of my citation counts he welcomed this
procedure as an objective alternative to what he was used to apply.
He gladly opened his "drawers" to let me take all athletes data
and do a reanalysis based on published plus unpublished
athletes. In this study subsamples of athletes differing in
numbers of citations were analysed seperately (similarly
to what was shown above with Tables 2 and 3). A pronounced
eminence correlation emerged.
Secondly, Nienhuys informed sci-skeptics about the low overall
G% of Gauquelin's unpublished sample as compared to his
published sample inviting the readers to draw their own
conclusions - and he could only expect his readers to draw wrong
conclusions (which he apparently wants to see spread) with not
informing them about essential details needed to draw
correct conclusions (which he probably deters). Nienhuys did not
inform readers about the UNPUBLISHED athletes' LOWER LEVEL OF
SUCCESS. G% difference between unpublished (=less eminent)
versus published (=more eminent) athletes MUST be obtained, if
Gauquelin's Mars + eminence hypothesis holds. The observed
difference is NOT CREATED but only INFLATED by Gauquelin's above
mentioned seductions.
A third neglect by Nienhuys must be set straight: He never
informed readers about the effects on eminence correlation
by my pooling Gauquelin's published with unpublished athletes.
I just quote from my paper (with some linguistic amendments):
"The crucial question remaining is as follows: Could the kind of
bias noted in Gauquelin's procedure invalidate the outcome of
the present study? Could an artefact carried over from original
meterials raise the risk for wrong conclusions? ...
[Gauquelin's] omission of athletes from experimental samples had
two effects: (a) It served to inflate the level of
kS-proportions [G%] overall. But also, (b), it weakened the
eminence effect. Our merging of unpublished with published data
did repair (i.e. lowered) the overall eminence level. But at the
same time it served to repair the eminence slope, i.e., to make
it steeper. Gauquelin's selection bias, therefore, does not
weaken the conclusion that Mars'position and the athletes'births
are statistically related. Paradoxical though it may seem, this
claim has been corroborated due to this bias: Correcting for
selection bias by pooling all data INCREASED empirical support
for the stronger version of this claim: the data have overcome,
IN SPITE OF DISTURBING EFFECTS OF BIAS, the higher
methodological hurdle."
---------------------------- 6 -------------------------------------
Before carelessly attributing to Gauquelin's (and my own) work
"methodological flaws", "file drawer effects", "rampant
significance reporting", significance fetism", "lots of tests -
one was significant", "no prior hypothesis", "post hoc
interpretations", "anomaly-mongering" - look at the above Tables
notably Tables 2 and 3. If you are suspicious of what is in the
right columns, keep to the left columns with the skeptics results.
This, hopefully, will foster thoughtfulness.
Those scientists among ourselves calling themselves 'skeptics'
should be more inclined than the remainder of us to reconsider
skeptically, when faced with unexpected evidence, their own
views. Once observational data demand it they should be ready
to admit that they were wrong, at least as ready as
"ordinary" scientists like Gauquelin who found and unearthed,
with utmost rigor, in a dust heap despised by the community what
might be considered - by an advanced posterity - as a real grain
of gold. Gauquelin was faced one day with unexpected evidence,
he was surprised by a replication failure in his "heredity"
research (he could not replicate G% similarities between parents
and their children), and he had this to say:
"The samples [my former and my recent sample] seemed identical in
every way, yet the first gave results strongly in support of the
hypothesis while the second gave results almost as strongly
contradicting the hypothesis ... Despite the disappointments, I
am pleased about one thing: If my latest work has created doubt,
then this is the best that can happen in science. As Bertrand
Russell has written, "Not to be absolutely certain is, I think,
one of the essential things about rationality".
(Gauquelin's heredity hypothesis did not find empirical support in
my studies either. His basic finding, however, (the one now
replicated by CFEPP) is totally independent of the heredity
construct which was part of Gauquelin's explanatoty model).
The Dutch skeptics (de Jager, Koppeschaar) had already
proclaimed in public to be able to "unmask" (as they said) the
Mars effect as an unrecognized artefact caused by seasonal/
diurnal priodicities. It happened that they failed entirely. If
they had stated their failure frankly and if Jan Nienhuys had
reported about their failure in frank manner for us electronic
readers I would compare him/them, regarding sincerity, with
Michel Gauquelin. I am still ready to do so and to withdraw my
suggestion to read Jan Nienhuys' messages more skeptically as
soon as he would refrain from wrongly reporting about Gauquelin
data and procedures and if he would show himself, instead,
convincing signs of skepticism regarding his own stand.
E-Mail Fredric L. Rice / The Skeptic Tank