The interesting bit is that I had a conversation with my syntax professor yesterday about questionable data in articles and he says that he often does surveys in the various undergraduate courses he teaches on questionable data - and usually 1/3 of the class doesn't like whatever sentence is supposed to be ungrammatical. Which is good, since it means whatever theory it is fits some portion of the population.
Now this is the part I don't like - if your theory only works for 1/3 of the population, and you're trying to make a "general theory of $foo", how general are you really making it? Particularly if you're going for "universal constraints" on language, which is usually what most of these guys want to do. Le sigh.
This is somewhat akin to a conversation I had with a linguistics guy who came from microbiology that I got to talk to up at UPenn. Basically, in linguistics, just as in biology, the data is usually messy. Biology, however, is perfectly fine with saying, "Woo. Okay, our data is messy. We can explain about 90% of it....and the rest of it we don't know what to do with right now. But the 90% we can explain always works." Or, if it's really controversial, maybe they'll say, "Well, this works about 30% of the time. No, we don't know why. But at least it works for that much."
Linguistics,, on the other hand, seems to have the push towards, "Um...it works 100% of the time! See how it works! Thus we have a theory about the general $foo of language! Wait, don't look at that data over there behind the curtain! Noooooooooooo.....". Though even that is better than, "See how it works for 100% of the data. See this questionable data that it works for!" "Er, but it doesn't always work on that questionable data..." "Yes, it does! Silence, infidel!"
Now, I admit the theoretical areas are more prone to this sort of behavior than the experimental (in fact, I've rarely seen the experimental guys do something quite this egregious - but maybe we just have a good department). Which is why I'm trying to push the gathering of data from at least some portion of the population in my theoretical jaunts. Much better, of course, would be to do a very large survey and some experimental measures to boot - but at least we need to get away from "Well, the author of the paper has these judgments and he's a native speaker, so there you have it."
Graaarg. Okay, my ranting is done.
By the by, for those who were wondering about the exact nature of the sentence in question - the what lies she tells bit was supposed to be an embedded exclamative and it was supposed to be completely ungrammatical when the upstairs "emotive" clause was negated: Jack is not surprised at. (Compared to when the upstairs "emotive" clause is not negated: Jack is suprised at what lies she tells. Which I do admit sounds more natural, but I was wondering if it was due to context or plausibility or some such, rather than straight up ungrammaticality.