Unfortunately, I’m not convinced by your conclusion. Academia pays fairly well, but not doctor or lawyer money. The big money is in speaking and consulting. Gino made such a fortune during her period of fame, I wouldn’t be surprised if she’s coming out solidly ahead in net.
Harvard pays B-School faculty very well. She had either a named professor or a chair, which will pay $500k annually at a minimum. She is about 48 years old, so 15 years (minimum) until retirement. What she gave up was massive, and her career prospects are poor, at best.
"It takes more effort to fake results than it does to create real ones."
There's a corollary in building stuff. It's very often more expensive to do stuff wrong than to follow the instructions and do it right. For one thing, the industry is built around the codes and doing it right; if you know where all the parts are at the local depot, it's like a kit and you simply assemble it.
We discuss instrumental ethics in my ethics class. They give me the heebie jeebies because it relies on people knowing they can be observed. So what happens if they don't think they'll be observed?
Data Colada (who caught it) spun up after the initial papers were published. They were among the first to systematically look for cheating in publishing. Had they been around before she was publishing it might have been a deterrent. Or not. I know I have changed some of my research methods due to them. Not because I was cheating before, but I want to create a clearer paper trail of how I got there.
When I was an assistant professor, I wrote a humorous paper called "How to cheat in college". It detailed how Professors catch cheating, and what you can do to avoid it. The last paragraph calls out "with this much effort to cheat, it might be quicker and easier just to do the assignment honestly". Which was the point. The associate provost in charge of innovative teaching complained about it. She didn't read it, just skimmed it. So we quietly pulled it down from web-sites.
Wow, this (and your following comments) really make me wonder who much real data is manipulated to meet a market deadline or a safety standard? I say this as the former owner of a Ford Pinto.... but I also remember my dad giving up cigarettes way back in the mid-50's, when the negative research first leaked out. That research was subsequently buried and "replaced" with other data for, what, 20 years or more?? And then there was the Oxycontin data, showing it was "safe"....
so I think your writing here is both explanatory and damning, along with being really important.
And while it's nice to know there are plenty of nerds out there to catch cheaters manipulating computer date, it would be even better if you wrote a "Spotting Deceptive Data for Dummies" manual for us non-nerds.
There is an essay by Dennis Goia on the Pinto story. He's now a retired academic, but prior to academia was a product recall manager for Ford. He didn't approve a recall because he didn't at the time think it was a systematic failure. He mentions he gave his pinto to his sister, thinking it was safe, not dangerous. I use the case in my ethics class.
I'll think about the "spotting deceptive data for Dummies", although there it's tough not to go into the weeds with it!
One correction to my post. I was re-reading Data Colada last night on a different topic and saw they posted an article on when they suspect "p-hacking" (changing the data to get the results to display significance).
I argued rather than alter data, just delete a few data points that go opposite what you hope to find. In a large study, dropping a few would be impossible to detect...or so I thought. Data Colada nerds have a way to raise a red flag about it. It's not proof, but it raises an eyebrow. 🤔
In laymen's terms, "p < .05" means you're confident to the 95th percentile that your analysis is correct. It's the norm, and prestigious journals don't often publish results higher than that, at least not for the main contribution. The Data Colada nerds examined hundreds of studies to look at displayed results. 80% of those around .05 are a smidge less than .05, but only 20% are smidge higher. That should be closer to 50/50. The easiest way to p-hack a "p < .051" to "p < .049" is to drop a variable from the analysis. They cannot prove you did it, but that "p < .049" raises a red flag.
Those nerds are good! 😬
As I shared with Phil below, in a current study I dropped a variable, but it lowered my results, not improved them. I did so because I thought it was confounded, that another reason explained it better than ours. Dropping confounds is the norm, but a nogoodnik could drop a problematic data point and claim it was a confound. 😡
First, a preemptive apologies to all scientists, academics and people's in general for my expressions below 🫣
Looking around at the academic and scientific fraud, which is probably tiny as a percentage of work (guess 0.1% ??), brings me back to my long held view that a significant amount of science research stems from:
A. An over abundance of really smart educated folks in 2025 vs say 1955.
B. An under abundance of "new things".
Failed wizzicist me, who had seminars with Heisenberg, de Broglie and Dirac at geek school half a century ago, has felt ...not much is new since a.century ago. I surmise that a lot of research is "filling in the deductive box". And some inductive research to enlarge a box.
But, what if there truly is just too much scientists in supply? And does that lead by competition survival, unethical conduct?
Plagiarism is to me, low brow stupidity. Data manipulation is clearly fraud. And I think there are subtle variations on this, like having good data but, deliberately leaving out non-agreeable data, defining success in a bias way as well.
It used to be more common, I think, to have reproduction of results, verification of them.
Again, just my hypothesis on the science and academic fraud.
I've read his books. His hypothesis is utterly rational and very unpleasant so folks don't like it. It explains our current situation better than anything else I've found.
I'm not a scientist. I don't know squat about all that stuff. I know his hypothesis answers the fundamental questions. People don't like his work because it's unpleasant with dire portents.
"who had seminars with Heisenberg, de Broglie and Dirac at geek school half a century ago,"
I was going to call you out for that fabulism, until I checked and found they died in 1976, 1987 and 1984, respectively, making your claim (sorta) plausible.
So how did you pull off this monumental feat of having seminars with the 3 founding fathers of quantum physics?
1974 or 75 but I can't recall anymore the year. They filled the main lecture hall blackboard with all kinds of equations and mostly beyond me. I also had a seminar (12 of us) with Aaga Bohr. Neil's Nobel son.
It also helped convince me I didn't have the chops. In the movie Oppenheimer, Bohr asks Oppy, a clumsy experimentalist, do you hear the music? He did. I didn't, at least mathematically.
One of our Mothership friends, Adam Solomon, is a brilliant practicing theoretical physicist. Working on quantum gravity!
I know just enough about physics (from introductory college phyiscs courses in college) to know how heavily math dependent is physics. In the '80s I acquired the book "In Search of Schrödingers's Cat" (still available on Amazon) that describes to laymen like me, the development of quantum physics from the late 19th and early 20th centuries, culminating in the discussion of that famous cat. A fascinating read.
I'm 71 next month. If you consider I was in college early 70s, it was only 60 ish years earlier when Einstein and Bohr did their work. Geek school time had many notables to see. Feynman of course, Gell-Mann, Shocking-Shockley.
Jay wrote: "Many of us create our initial data files in Microsoft Office, which contain both a “created date” and a “last modified date”. Which means if you intend to fake your data, you must create the fake document the same time as the real document"
But, isn't it very easy to use PowerShell (windows) or Touch (mac) to change those dates to whatever you want?
I'm more of a Mac and Linux IT person than a Windows person, but I bet there is some utility out there that would do the job, if not in Powershell itself. On a Mac, it would be easy to do from the command line.
In fairness to our mother tongue, other languages have irregular features that just have to be learned by rote. Many Indo-European languages (other than english, interestingly enough) have gendered nouns -- 2 in the Romance languages, 3 in Germanic, Russian and other languages. In German specifically, most nouns have irregular plural forms that simply have to be learned along with the noun.
Then there are the languages of the East like Mandarin Chinese that use pictographic systems, whose symbols have to be learned, possibly aiding pronunciation but often not. Kurt has written about this.
And that doesn't even address Native American languages, like Navajo, a language so difficult it was used as radio code by the US during WWII.
Nevertheless, my sympathies to those for whom English is not their first language. 🙂
One of the arguments against English "spelling reform" is that it would greatly complicate future reading of most of the world's existing body of writing in English.
On the other hand, computers could convert pre-reform texts to the post-reform system much more easily than humans could have done it a generation ago. On the left foot, though, this would provide an opportunity for the content of the texts to be changed in material ways. This sometimes happens now when new physical editions are published, and it's also a real problem with online writing.
Speaking of college, Daughter B phoned yesterday to tell us about the field work program she did in the Bend, OR, area over the weekend. It was a program for about 60 undergraduates to learn the skills they'll need if they're going to be researchers or practitioners in wildlife, forestry, and other kinds of outdoorsy sciences.
They had to set up a camp. (She already knew this stuff from Scouts.) They dissected some deer carcasses - hers died from malnutrition - and learned to take samples to test for Chronic Wasting Disease. They learned to change a tire and do basic maintenance on a truck. They set up trail cameras, did insect-population sampling, practiced for a program about listening for bat sounds, and several other things.
She was really excited about it. This semester has been mostly indoor classes - data studies, programming, more chemistry - but she said she'll have more fun things next term.
Be a doctor (MD, PhD, doesn't much matter) with a strange hobby, such as handling scolopendrids (the big, scary centipedes). Get injured doing the hobby. Include one's injuries, writing about oneself in the third person, in a compendium of case studies.
Even the biggest, scariest centipede is unlikely to do a basically healthy adult human much damage. Plausible reports of death from centipede envenomation seem limited to the very young (<5), very old, and already infirm. Still, I'm glad to live up north where our little centipedes (house and stone) are positively cute by comparison.
I am pleased she is excited about it, and that she was able to handle the camp. With a scouting background I had no concerns about it, but she has an advantage by being able to do so.
I required my children to change a car tire as part of getting a driver's license. Too many don't know how.
You just need a better lug wrench. The one supplied with the car is lousy; even I can't loosen nuts with that thing. You need cross type w/24" span. Makes it easy.
I never learned to change a tire, fortunately never needed to. But I did know how to jump-start a car. My mom gave me a nice set of jumper cables with the correct instructions to do it safely written out step by step. Came in real handy a few times, not just for me but for other people.
Oh, yes. And then there are the local customs. When one sees a group of people pushing a car that is a little bit stuck, one joins in. And then there's parking "dibs," of which I approve, provided the amount of snowfall is large enough to make it necessary to protect the amount of labor it takes to shovel out such a space from being promptly hijacked by the lazy and selfish.
I rarely met a lug nut I couldn't loosen. Back in my younger days :(
I change a lot of tires on vehicles, trailers, farm equipment. I discovered they now make cordless impact guns that will take off just about any lug nut, and the prices are extremely reasonable for non-professional grade use. Now that I think about it, I have steadily built up an arsenal of such equipment over the past few years. You may think that this implies a steady loss of muscle strength and stamina. And you would be correct ;)
When my 36" breaker bar didn't do the trick, we have a 10' pipe to put over the breaker bar handle that always did the trick. But I think my grandchildren are the only family members strong enough to lift that pipe anymore :(
I've changed tires before. But when our family SUV had a flat tire, I had no patience with the spare stored under the body. I wimped out and called AAA.
The last time I changed a tire was a few years ago, in rental Jeep Grand Wagoneer, in full tuxedo on the way home from a wedding. I had help from my mechanical engineer son-in-law. My new daughter-in-law was so impressed she filmed and posted to whatever social media she uses. As you can see, we had the full contingent of family in the car. The spare was under the vehicle and quite a challenge to get without ruining the tux!
That is high up on my list of the most irritating things. Hmmm, I wonder if there is a parody of Sound of Music "These are a few of my favorite things"?
Regarding tires, this boomer was clueless in the opposite way. I was recently flagged down by a young man in a Tesla, out of cell range, with a flat tire. I had no idea Teslas (and apparently other cars) don't have spare tires. He didn't know one way or the other so it took looking through the manual to convince me. I wouldn't be comfortable with no spare outside of a reasonably large metro area.
If your car has no spare it had better have darn good tires that can run flat at least for a relatively short distance. Sounds like a(nother) good reason not to get a Tesla.
As far as I know, no pump. The car has a very reassuring "call Tesla" button, which is useless out of cell range. That was another thing I didn't understand - why isn't it connected via StarLink? Even as a tandem with cellular?
I would ask my son. But he just traveled to Japan for advanced language studies with the Navy. To my great shock, he turned his Tesla back in to the dealer before leaving. He loved that car. But I suspect something changed in his passionate love of all things Musk. He's too busy with his studies to bother asking him what happened.
No idea if it applies, but there was some talk that Teslas should be banished from all sensitive US national security installations once Musk made his deal with the Chinese government. From that point on, it was understood that all the cars’ cameras and sensors would collect data that would be shared with the mainland PRC government…
Not only years of Scouting, but five years in the Marine Corps. I think she has more real-world competency than the average college student. She had also changed a tire in the past, but not recently, I suppose.
When Thor took Fieldwork Methods at UNC-Charlotte, he was the only one in his class with any experience at outdoor skills. I wonder if students at Oregon State are more outdoorsy in general.
B said it was the first time the school has done this program, so they must have seen a need for it. She has applied for a summer internship in Desert Ecosystem Protection in eastern Oregon. We haven't heard yet if she expects to be hired.
I missed the discussion yesterday so I really appreciate this post, Jay. I wonder about the subject of dishonesty broadly—does studying it create less honesty? That is, does the subject matter have an influence on the researcher? Starts to feel like quantum physics…
Whew! Good question. As I wrote in a reply to Phil, my students joke how I know how to cheat! 😬 I dunno why I see these things, other than being a nerd. And somewhere I saw a paper abstract saying Ethics professors have lower ethical values than others! 🤦♂️
In general I'm normally not a fan of "instrumental ethics" (we do the right thing because it benefits us personally to do so), but I think in academia it makes sense. If you get caught cheating in research, you can get fired; worse, other colleges will hesitate to hire you. Your career is over. Because once you're tenured this is a good life, that's a high price to pay for what will be (for most) a minimal gain. But for her, she shifted from an excellent college to a superstar one; she got a big gain from it.
I like learning about cheating, not to cheat, but how to avoid making errors. In excel, sometimes you code something initially with a label, just to understand it, then you go back ana analyze what you have. If you think there's something there you then create the numerical value to capture it. But, autocorrect can fill in the wrong label; you have to be alert to catch it. So now I start every label with a different letter, to avoid autocorrect. I am sure in my earlier work I have an autocorrect error somewhere in the data. I hope not, but it could happen.
In my field journals now want you to post your source data and the code for your analysis. Posting the code is easy; you click on a "log file", run the analysis, then save it. Although I run a clean version, because I often "what if" a ton of ideas during analysis, 95% never matter. I myself save the data as a text file, because almost all the major packages can import it into their software easily. Although, as I noted, doing so makes it harder to catch cheating.
Autocorrect is a type of AI, is it not? More evidence that AI is not always helpful or trustworthy. Seems to me that tricks like yours, to thwart AI in its stupid attempts to mess with one's conscientiously prepared work product, will become popular.
My students laugh that I appear to have a gift for understanding how to cheat. 😬
There are things I do to reduce errors that could look like cheating.
In a recent study I coded what type of action companies took in a certain announcement. I had my co-authors use the same codes to classify the data, to see if they matched mine (they didn't know how I coded them). We had over 95% agreement; the difference were only in ones that had multiple things going on. We discussed those few, coming to agreement on what they should be. When possible, we look for ways to "automate" this; not to save time but to get consistency.
The toughest one for me is when to remove "outliers" in the data. It is always a judgement call. The idea is if you can argue one data point has an issue. In an event study, we do a search for confounds (two events happening at the same time). When we find one we drop it from the sample. But if you want to cheat, you might drop it from the sample to improve the results. It's a judgement call. In a recent study we had another company (not in the study) announce it was discussing an acquisition with a company in the study. The stock price increased over 20%. We excluded it, thinking the acquisition was what drove the stock price change (prior studies show firms being acquired have large stock price gains). Had we kept it in the main effects would have been strengthened. The average gain would have climbed from 1.55% to 2.47%, which is a huge gain. It also would have dropped one of the supporting hypotheses from being significant. Now, we dropped this firm from the sample at the data collection stage, not at the analysis stage. But I added it back in case an editor ever asks why we dropped it.
Good morning. 56 degrees here, cloudy with some rain later and highs in the 60s. Interesting weather for Columbus’s annual PGA tour event, the Memorial golf tournament, founded and hosted by Columbus golf great Jack Nicklaus. Practice rounds today, the first round is tomorrow. Per local custom, it almost always rains at some point during the tournament.
The mothership is covering Trump’s threat to pause the international college students program, impacting not just Harvard, with who the administration has been feuding, but also colleges great and small across the nation.
The FP, meanwhile, has this annoying habit of auto-playing a video whenever I select the Front Page item from my Substack feed.
Our local Y for over 20 years operated our community's outdoor pool. This year they decided to take over running it. So the cold weather this week has been amusing, because they will have poor attendance. But I suspect the weather will warm up soon. But you want a mini heat wave two weeks before the pool opens so it will warm the water.
Although walking the Indy mini marathon in this weather was great!
Good morning, Phil. 57 here, 100% chance of rain, might clear up in the afternoon and be in the mid-70s.
UNC-Charlotte has a lot of foreign students. I remember when Beau graduated from the business college, there were nearly 100 guys from India getting Masters in computer science. They probably all went to work for the banks and now play cricket on the weekends out here in the 'burbs.
Ohio state has a lot of international students as well. Trump's actions will reverberate across many universities across the country, almost all of which don't have Harvard's multibillion dollar endowments.
I know why Trump is doing it, but I disagree. I think the solution is to hire more people to vet all immigrant students, and deny those who are problematic. Err on the side of caution. Right now we don't.
With no small irony, a big chunk of my professional career (including academia) occurred because a Chinese student was denied a visa just as the fall semester started. The student had a graduate assistant position, and computing services didn't want it to go unfilled (use it or lose it). They called the business school; the qualifications were 'enrolled' and "has a pulse". I met those qualifications.
I don’t disagree with vetting. I disagree strongly with abruptly ending, even suspending, international student admissions across the board. This is Trump’s “bull in a china shop” approach, which is harmful even when I might otherwise agree.
My sentiments exactly. There are many ideas that he is pursuing that I favor, but am disgusted at his lousy means to the ends. Most of it will either come to naught or prove completely counterproductive—especially policies that get a bad rap due to incompetent and/or brutal implementation.
Very interesting comments, Jay. I appreciate your going into this in more detail. A phrase that jumped out at me was "Francesca Gino, a celebrated behavioral scientist".
Do you think the fact that Dr. Gino is a "celebrated" "scientist" is a significant factor in the decisions she made when the data nerds raised questions about her research?
What kind of study might be done to evaluate whether a person identifying as a "scientist" is more or less likely to treat accurate data collection as a matter of fiduciary responsibility, rather than in a more loosey-goosey, outcome-oriented sort of way? What about a person who is identified as "celebrated" or "celebrity" or otherwise outside the run-of-the-mill?
The discussion yesterday both here and at the Mothership reminded me of a podcast Jonah Goldberg did with a teacher/writer in the criminology field. The guest kept proudly self-identifying as a "scientist," to the extent that it really caught my attention as a listener. It seemed to me that he was more like a data researcher, which is a worthy occupation ... but is it the same as a SCIENTIST?
I think you might be right: she wanted to produce something 'Big" once she was at Harvard, and this research wasn't it. Professor Gino was graduated from the University of Triente; which is a pretty good university, but not at the Harvard level. For her to get selected within 6 years to jump from UNC to Harvard meant they thought she had "Star" potential. That's a pretty big jump up. At least two of her co-authors were "Stars" as well (Bazerman and Arielly). So she was expected to produce big results, field changing results. As I said, when I saw the effect sizes I was shocked.
This is a key point. Modest changes ought to produce modest effect sizes. Introducing a signature to a firm where there was none previously, that might produce a big change. But shifting it from top to bottom on the form? It felt modest to me.
I didn't say anything when I first saw it because it wasn't my field, so being critical would have gotten me labeled sexist, or a conspiracist nutter.
Scientist is a term that doesn't have real meaning. I am considered a "social scientist" because Management is considered a social science, albeit set in a professional school. I only use the term to set myself apart from physicists and the like. Data scientist is a very popular term for those who study how to gather and handle extremely large data sets. I don't like to use the term because it has "prestige" connotations.
Thanks. I agree that "scientist" is a term that is missing a firm definition. I brought that up yesterday at the Mothership, and some people thought I was making a point about policy or about people, rather than about what words actually mean.
There’s something in there related to the distinction between gaining prestige and asserting dominance. In the dominance hierarchy, the dominant ones, the kings of their own hills, demand that others approach them obsequiously.
Those who gain prestige are largely given it by popular acclaim. Prestige isn’t when the bearer demands fealty from others. In small-scale tribal societies, prestigious individuals, like the village elders, tend to be modest and self-deprecating, while they are widely acknowledged as especially skillful in specific areas of their expertise.
Dominance is asserted and requires the use of force or threat of violence. It is the ancient animalistic trait with roots going back to the simplest creatures, used to determine who gets the first, best access to scarce resources. The top of the academic pecking order looks a lot like a dominance hierarchy to me.
I get the distinction you're making between "dominance" and "prestige." That's from the book you've been reading!
I would suggest that there's a mix of the two in the academic situation. Maybe first a person acquires prestige, and then the person gets into a dominance hierarchy.
Also, I wonder if one of the elements here is Dr. Gino's being a woman. Did she think she would be protected from consequences because of that?
Just to clarify: Not sure exactly what I had in mind, but I think the dominance/prestige distinction was more relevant to the podcast you mentioned where the person had to keep saying “I’m a scientist.”
That caused flashbacks to the climate “debates” where you’ve had *some people* declare themselves not only “climate scientists”, but also arbiters of who else on earth gets to call him/herself one, too. Crazytown.
Interesting point. Do you think a person proclaiming himself to be a SCIENTIST is asserting dominance over the non-scientist person, such as a journalist? If I correctly understand the distinction, "prestige" is conferred on a person by others, rather than claimed and defended.
Maybe. Or maybe she was encouraged to think that way over her career.
I imagine it’s one of those things that happens to people who start off by bending the facts a bit and then get used to it, once they find out they can get away with the behavior. I’m sure it’s like the progression of Evil (to cite Baumeister’s book): the subject tells herself she’s doing important work for a good cause, and is sure the outcome is correct, but just needs a bit of assistance until the *right* study design becomes clear.
Rule-bending and -breaking are things we get habituated to, though, and once we’ve reduced our inhibitions the initial slips become an easy slide. The subject believes her significance and importance constitute a just cause in itself, and so on.
As with the distinctions between dominance and prestige, there are grey-zone boundaries where the two get hard to tell apart. Thus it is for a striving academic—fed on years’ worth of people with prestige telling them how important they are—to begin to see their own efforts as too important to let accurate data reporting get in the way, and so on.
Data Colada found four separate papers where there were issues. So once she began to fall, she fell all the way.
Interestingly enough, the first paper was one where the "big name" author was Dan Arielly (a big name) was the one being checked out. He claims he didn't modify the data, it was given to him as such, and he didn't bother to check to see if it had been modified. But she was a co-author. Did she learn from him, or was she cheating back then.
But the later papers all showed the same pattern: duplicate influential data points to improve a finding from no significance to significance.
"A renowned Harvard University professor was stripped of her tenure and fired after an investigation found she fabricated data on multiple studies focused on dishonesty.
The irony. It burns.
I've followed this story on multiple sites, substacks and pods. Its really quite incredible.
Unfortunately, I’m not convinced by your conclusion. Academia pays fairly well, but not doctor or lawyer money. The big money is in speaking and consulting. Gino made such a fortune during her period of fame, I wouldn’t be surprised if she’s coming out solidly ahead in net.
Harvard pays B-School faculty very well. She had either a named professor or a chair, which will pay $500k annually at a minimum. She is about 48 years old, so 15 years (minimum) until retirement. What she gave up was massive, and her career prospects are poor, at best.
Yeah, but rent on a used refrigerator box in Somerville is $25k/mo, maybe more if it’s near a popular fire drum.
RE: university prof fabricates data in studies about dishonesty
I'd be lyin' if I said anything other than you just can't make this sh*t up.
Fascinating post. Thanks
"It takes more effort to fake results than it does to create real ones."
There's a corollary in building stuff. It's very often more expensive to do stuff wrong than to follow the instructions and do it right. For one thing, the industry is built around the codes and doing it right; if you know where all the parts are at the local depot, it's like a kit and you simply assemble it.
"It takes more effort to fake results than it does to create real ones."
"felt she just needed a little help to get the incredible results she wanted."
What do they teach them in these schools? Neither logic nor ethics, apparently.
We discuss instrumental ethics in my ethics class. They give me the heebie jeebies because it relies on people knowing they can be observed. So what happens if they don't think they'll be observed?
Data Colada (who caught it) spun up after the initial papers were published. They were among the first to systematically look for cheating in publishing. Had they been around before she was publishing it might have been a deterrent. Or not. I know I have changed some of my research methods due to them. Not because I was cheating before, but I want to create a clearer paper trail of how I got there.
When I was an assistant professor, I wrote a humorous paper called "How to cheat in college". It detailed how Professors catch cheating, and what you can do to avoid it. The last paragraph calls out "with this much effort to cheat, it might be quicker and easier just to do the assignment honestly". Which was the point. The associate provost in charge of innovative teaching complained about it. She didn't read it, just skimmed it. So we quietly pulled it down from web-sites.
"The associate provost in charge of innovative teaching complained about it."
Lol!
Wow, this (and your following comments) really make me wonder who much real data is manipulated to meet a market deadline or a safety standard? I say this as the former owner of a Ford Pinto.... but I also remember my dad giving up cigarettes way back in the mid-50's, when the negative research first leaked out. That research was subsequently buried and "replaced" with other data for, what, 20 years or more?? And then there was the Oxycontin data, showing it was "safe"....
so I think your writing here is both explanatory and damning, along with being really important.
And while it's nice to know there are plenty of nerds out there to catch cheaters manipulating computer date, it would be even better if you wrote a "Spotting Deceptive Data for Dummies" manual for us non-nerds.
There is an essay by Dennis Goia on the Pinto story. He's now a retired academic, but prior to academia was a product recall manager for Ford. He didn't approve a recall because he didn't at the time think it was a systematic failure. He mentions he gave his pinto to his sister, thinking it was safe, not dangerous. I use the case in my ethics class.
I'll think about the "spotting deceptive data for Dummies", although there it's tough not to go into the weeds with it!
One correction to my post. I was re-reading Data Colada last night on a different topic and saw they posted an article on when they suspect "p-hacking" (changing the data to get the results to display significance).
I argued rather than alter data, just delete a few data points that go opposite what you hope to find. In a large study, dropping a few would be impossible to detect...or so I thought. Data Colada nerds have a way to raise a red flag about it. It's not proof, but it raises an eyebrow. 🤔
In laymen's terms, "p < .05" means you're confident to the 95th percentile that your analysis is correct. It's the norm, and prestigious journals don't often publish results higher than that, at least not for the main contribution. The Data Colada nerds examined hundreds of studies to look at displayed results. 80% of those around .05 are a smidge less than .05, but only 20% are smidge higher. That should be closer to 50/50. The easiest way to p-hack a "p < .051" to "p < .049" is to drop a variable from the analysis. They cannot prove you did it, but that "p < .049" raises a red flag.
Those nerds are good! 😬
As I shared with Phil below, in a current study I dropped a variable, but it lowered my results, not improved them. I did so because I thought it was confounded, that another reason explained it better than ours. Dropping confounds is the norm, but a nogoodnik could drop a problematic data point and claim it was a confound. 😡
First, a preemptive apologies to all scientists, academics and people's in general for my expressions below 🫣
Looking around at the academic and scientific fraud, which is probably tiny as a percentage of work (guess 0.1% ??), brings me back to my long held view that a significant amount of science research stems from:
A. An over abundance of really smart educated folks in 2025 vs say 1955.
B. An under abundance of "new things".
Failed wizzicist me, who had seminars with Heisenberg, de Broglie and Dirac at geek school half a century ago, has felt ...not much is new since a.century ago. I surmise that a lot of research is "filling in the deductive box". And some inductive research to enlarge a box.
But, what if there truly is just too much scientists in supply? And does that lead by competition survival, unethical conduct?
https://www.acfe.com/fraud-resources/fraud-examiner-archives/fraud-examiner-article?s=September-2022-Medical-Fraud
https://www.drugdiscoverytrends.com/biomedical-research-integrity-scandals/
Who knew the fraud detective field was so large! Sadly so.
https://en.m.wikipedia.org/wiki/List_of_scientific_misconduct_incidents
Plagiarism is to me, low brow stupidity. Data manipulation is clearly fraud. And I think there are subtle variations on this, like having good data but, deliberately leaving out non-agreeable data, defining success in a bias way as well.
It used to be more common, I think, to have reproduction of results, verification of them.
Again, just my hypothesis on the science and academic fraud.
"But, what if there truly is just too much scientists in supply? And does that lead by competition survival, unethical conduct?"
Turchin in a nutshell.
https://en.m.wikipedia.org/wiki/Peter_Turchin
I was unaware of Turchin and his work. Cool.
A Hari Seldon for our age.
I've read his books. His hypothesis is utterly rational and very unpleasant so folks don't like it. It explains our current situation better than anything else I've found.
I like that he used retrospective history and events to test his models
I'm not a scientist. I don't know squat about all that stuff. I know his hypothesis answers the fundamental questions. People don't like his work because it's unpleasant with dire portents.
"who had seminars with Heisenberg, de Broglie and Dirac at geek school half a century ago,"
I was going to call you out for that fabulism, until I checked and found they died in 1976, 1987 and 1984, respectively, making your claim (sorta) plausible.
So how did you pull off this monumental feat of having seminars with the 3 founding fathers of quantum physics?
I have no science background, but was fascinated with these and other characters in the book "When We Cease to Understand The World".
They came to an honorarium for Professor Victor Weiskopf, a Neil's Bohr student.
https://en.m.wikipedia.org/wiki/Victor_Weisskopf
1974 or 75 but I can't recall anymore the year. They filled the main lecture hall blackboard with all kinds of equations and mostly beyond me. I also had a seminar (12 of us) with Aaga Bohr. Neil's Nobel son.
It also helped convince me I didn't have the chops. In the movie Oppenheimer, Bohr asks Oppy, a clumsy experimentalist, do you hear the music? He did. I didn't, at least mathematically.
One of our Mothership friends, Adam Solomon, is a brilliant practicing theoretical physicist. Working on quantum gravity!
Nice story!
I know just enough about physics (from introductory college phyiscs courses in college) to know how heavily math dependent is physics. In the '80s I acquired the book "In Search of Schrödingers's Cat" (still available on Amazon) that describes to laymen like me, the development of quantum physics from the late 19th and early 20th centuries, culminating in the discussion of that famous cat. A fascinating read.
RE: "In Search of Schrödinger's Cat"
That book was published in 1984, more than 4 decades ago. Schrödinger posited his cat in 1935, 90 years ago. Hmmm... let's see...
Through some non-Quantum figuring we learn that:
Avg cat single lifespan = 12 years
9 lives X 12 years = 108 feline life years
108 - 90 = 18 feline life years / 9 years per feline life =
2 cat lives left before we can quit searching for it and declare once and for all that the damn cat's dead.
I'm 71 next month. If you consider I was in college early 70s, it was only 60 ish years earlier when Einstein and Bohr did their work. Geek school time had many notables to see. Feynman of course, Gell-Mann, Shocking-Shockley.
You might enjoy Arthur Eddingtons books on Relativity as well. Abbott book Flatland is fun.
One of my all time favorite books is the Pulitzer winning “Making of the Atomic Bomb” by Richard Rhoades.
Jay wrote: "Many of us create our initial data files in Microsoft Office, which contain both a “created date” and a “last modified date”. Which means if you intend to fake your data, you must create the fake document the same time as the real document"
But, isn't it very easy to use PowerShell (windows) or Touch (mac) to change those dates to whatever you want?
At my University they disable users from PowerShell access! But if you have a home PC you could do that!. 🤔
I'm more of a Mac and Linux IT person than a Windows person, but I bet there is some utility out there that would do the job, if not in Powershell itself. On a Mac, it would be easy to do from the command line.
An honest dishonesty???
Changing Ye Old English post Middle Ages to stabilize the English language....🐾🐾🐾🐐🐐🐐🦉
https://open.substack.com/pub/colingorrie/p/why-english-spelling-will-never-make
In fairness to our mother tongue, other languages have irregular features that just have to be learned by rote. Many Indo-European languages (other than english, interestingly enough) have gendered nouns -- 2 in the Romance languages, 3 in Germanic, Russian and other languages. In German specifically, most nouns have irregular plural forms that simply have to be learned along with the noun.
Then there are the languages of the East like Mandarin Chinese that use pictographic systems, whose symbols have to be learned, possibly aiding pronunciation but often not. Kurt has written about this.
And that doesn't even address Native American languages, like Navajo, a language so difficult it was used as radio code by the US during WWII.
Nevertheless, my sympathies to those for whom English is not their first language. 🙂
One of the arguments against English "spelling reform" is that it would greatly complicate future reading of most of the world's existing body of writing in English.
On the other hand, computers could convert pre-reform texts to the post-reform system much more easily than humans could have done it a generation ago. On the left foot, though, this would provide an opportunity for the content of the texts to be changed in material ways. This sometimes happens now when new physical editions are published, and it's also a real problem with online writing.
Speaking of college, Daughter B phoned yesterday to tell us about the field work program she did in the Bend, OR, area over the weekend. It was a program for about 60 undergraduates to learn the skills they'll need if they're going to be researchers or practitioners in wildlife, forestry, and other kinds of outdoorsy sciences.
They had to set up a camp. (She already knew this stuff from Scouts.) They dissected some deer carcasses - hers died from malnutrition - and learned to take samples to test for Chronic Wasting Disease. They learned to change a tire and do basic maintenance on a truck. They set up trail cameras, did insect-population sampling, practiced for a program about listening for bat sounds, and several other things.
She was really excited about it. This semester has been mostly indoor classes - data studies, programming, more chemistry - but she said she'll have more fun things next term.
Cool
Sounds like D B could be amused by the following use of an advanced degree:
https://journals.sagepub.com/doi/full/10.1580/1080-6032(2001)012[0093:CE]2.0.CO;2
To summarize, this is the use:
Be a doctor (MD, PhD, doesn't much matter) with a strange hobby, such as handling scolopendrids (the big, scary centipedes). Get injured doing the hobby. Include one's injuries, writing about oneself in the third person, in a compendium of case studies.
Even the biggest, scariest centipede is unlikely to do a basically healthy adult human much damage. Plausible reports of death from centipede envenomation seem limited to the very young (<5), very old, and already infirm. Still, I'm glad to live up north where our little centipedes (house and stone) are positively cute by comparison.
I am pleased she is excited about it, and that she was able to handle the camp. With a scouting background I had no concerns about it, but she has an advantage by being able to do so.
I required my children to change a car tire as part of getting a driver's license. Too many don't know how.
I know how to change a tire, but have never had the strength to loosen lug nuts tightened by machine.
You just need a better lug wrench. The one supplied with the car is lousy; even I can't loosen nuts with that thing. You need cross type w/24" span. Makes it easy.
Amazon....
https://a.co/d/hjcWSk3
I have the best tire changing tool ever. It's called a AAA card.
yeah, but I don't like waiting.
Well, I don't have a car anymore, so that's moot now. Plus, my upper body strength is not what it used to be.
I never learned to change a tire, fortunately never needed to. But I did know how to jump-start a car. My mom gave me a nice set of jumper cables with the correct instructions to do it safely written out step by step. Came in real handy a few times, not just for me but for other people.
Chicago winter....jump start knowledge is survival training.
Oh, yes. And then there are the local customs. When one sees a group of people pushing a car that is a little bit stuck, one joins in. And then there's parking "dibs," of which I approve, provided the amount of snowfall is large enough to make it necessary to protect the amount of labor it takes to shovel out such a space from being promptly hijacked by the lazy and selfish.
Dibs is OK. But, only for night after work hours. No dibs during the day.
I rarely met a lug nut I couldn't loosen. Back in my younger days :(
I change a lot of tires on vehicles, trailers, farm equipment. I discovered they now make cordless impact guns that will take off just about any lug nut, and the prices are extremely reasonable for non-professional grade use. Now that I think about it, I have steadily built up an arsenal of such equipment over the past few years. You may think that this implies a steady loss of muscle strength and stamina. And you would be correct ;)
A nice, long breaker bar does wonders. A 36” one and you could put a lot of stout young men with basic manual ratchet sets to shame!
https://www.harborfreight.com/search?q=breaker%20bars
When my 36" breaker bar didn't do the trick, we have a 10' pipe to put over the breaker bar handle that always did the trick. But I think my grandchildren are the only family members strong enough to lift that pipe anymore :(
I've changed tires before. But when our family SUV had a flat tire, I had no patience with the spare stored under the body. I wimped out and called AAA.
The last time I changed a tire was a few years ago, in rental Jeep Grand Wagoneer, in full tuxedo on the way home from a wedding. I had help from my mechanical engineer son-in-law. My new daughter-in-law was so impressed she filmed and posted to whatever social media she uses. As you can see, we had the full contingent of family in the car. The spare was under the vehicle and quite a challenge to get without ruining the tux!
You're a better man than I am!
For sure quite a few would see it the other way - that you are the one with common sense!
That is high up on my list of the most irritating things. Hmmm, I wonder if there is a parody of Sound of Music "These are a few of my favorite things"?
Regarding tires, this boomer was clueless in the opposite way. I was recently flagged down by a young man in a Tesla, out of cell range, with a flat tire. I had no idea Teslas (and apparently other cars) don't have spare tires. He didn't know one way or the other so it took looking through the manual to convince me. I wouldn't be comfortable with no spare outside of a reasonably large metro area.
If your car has no spare it had better have darn good tires that can run flat at least for a relatively short distance. Sounds like a(nother) good reason not to get a Tesla.
Not even a pump? Acuras at least have pumps.
As far as I know, no pump. The car has a very reassuring "call Tesla" button, which is useless out of cell range. That was another thing I didn't understand - why isn't it connected via StarLink? Even as a tandem with cellular?
I would ask my son. But he just traveled to Japan for advanced language studies with the Navy. To my great shock, he turned his Tesla back in to the dealer before leaving. He loved that car. But I suspect something changed in his passionate love of all things Musk. He's too busy with his studies to bother asking him what happened.
No idea if it applies, but there was some talk that Teslas should be banished from all sensitive US national security installations once Musk made his deal with the Chinese government. From that point on, it was understood that all the cars’ cameras and sensors would collect data that would be shared with the mainland PRC government…
Good question for Elon. Who knows, not that he has stepped back from DOGE, he might even answer. 🙂
🙄
Not only years of Scouting, but five years in the Marine Corps. I think she has more real-world competency than the average college student. She had also changed a tire in the past, but not recently, I suppose.
That sounds exciting! Anything that’s hands-on and out in Nature would have to be more interesting than mere lectures.
When Thor took Fieldwork Methods at UNC-Charlotte, he was the only one in his class with any experience at outdoor skills. I wonder if students at Oregon State are more outdoorsy in general.
B said it was the first time the school has done this program, so they must have seen a need for it. She has applied for a summer internship in Desert Ecosystem Protection in eastern Oregon. We haven't heard yet if she expects to be hired.
I missed the discussion yesterday so I really appreciate this post, Jay. I wonder about the subject of dishonesty broadly—does studying it create less honesty? That is, does the subject matter have an influence on the researcher? Starts to feel like quantum physics…
Whew! Good question. As I wrote in a reply to Phil, my students joke how I know how to cheat! 😬 I dunno why I see these things, other than being a nerd. And somewhere I saw a paper abstract saying Ethics professors have lower ethical values than others! 🤦♂️
In general I'm normally not a fan of "instrumental ethics" (we do the right thing because it benefits us personally to do so), but I think in academia it makes sense. If you get caught cheating in research, you can get fired; worse, other colleges will hesitate to hire you. Your career is over. Because once you're tenured this is a good life, that's a high price to pay for what will be (for most) a minimal gain. But for her, she shifted from an excellent college to a superstar one; she got a big gain from it.
I like learning about cheating, not to cheat, but how to avoid making errors. In excel, sometimes you code something initially with a label, just to understand it, then you go back ana analyze what you have. If you think there's something there you then create the numerical value to capture it. But, autocorrect can fill in the wrong label; you have to be alert to catch it. So now I start every label with a different letter, to avoid autocorrect. I am sure in my earlier work I have an autocorrect error somewhere in the data. I hope not, but it could happen.
In my field journals now want you to post your source data and the code for your analysis. Posting the code is easy; you click on a "log file", run the analysis, then save it. Although I run a clean version, because I often "what if" a ton of ideas during analysis, 95% never matter. I myself save the data as a text file, because almost all the major packages can import it into their software easily. Although, as I noted, doing so makes it harder to catch cheating.
Autocorrect is a type of AI, is it not? More evidence that AI is not always helpful or trustworthy. Seems to me that tricks like yours, to thwart AI in its stupid attempts to mess with one's conscientiously prepared work product, will become popular.
Thanks Jay, for the interesting tutorial on how to cheat academically and not get caught. 🙂
My students laugh that I appear to have a gift for understanding how to cheat. 😬
There are things I do to reduce errors that could look like cheating.
In a recent study I coded what type of action companies took in a certain announcement. I had my co-authors use the same codes to classify the data, to see if they matched mine (they didn't know how I coded them). We had over 95% agreement; the difference were only in ones that had multiple things going on. We discussed those few, coming to agreement on what they should be. When possible, we look for ways to "automate" this; not to save time but to get consistency.
The toughest one for me is when to remove "outliers" in the data. It is always a judgement call. The idea is if you can argue one data point has an issue. In an event study, we do a search for confounds (two events happening at the same time). When we find one we drop it from the sample. But if you want to cheat, you might drop it from the sample to improve the results. It's a judgement call. In a recent study we had another company (not in the study) announce it was discussing an acquisition with a company in the study. The stock price increased over 20%. We excluded it, thinking the acquisition was what drove the stock price change (prior studies show firms being acquired have large stock price gains). Had we kept it in the main effects would have been strengthened. The average gain would have climbed from 1.55% to 2.47%, which is a huge gain. It also would have dropped one of the supporting hypotheses from being significant. Now, we dropped this firm from the sample at the data collection stage, not at the analysis stage. But I added it back in case an editor ever asks why we dropped it.
Good morning. 56 degrees here, cloudy with some rain later and highs in the 60s. Interesting weather for Columbus’s annual PGA tour event, the Memorial golf tournament, founded and hosted by Columbus golf great Jack Nicklaus. Practice rounds today, the first round is tomorrow. Per local custom, it almost always rains at some point during the tournament.
The mothership is covering Trump’s threat to pause the international college students program, impacting not just Harvard, with who the administration has been feuding, but also colleges great and small across the nation.
The FP, meanwhile, has this annoying habit of auto-playing a video whenever I select the Front Page item from my Substack feed.
Our local Y for over 20 years operated our community's outdoor pool. This year they decided to take over running it. So the cold weather this week has been amusing, because they will have poor attendance. But I suspect the weather will warm up soon. But you want a mini heat wave two weeks before the pool opens so it will warm the water.
Although walking the Indy mini marathon in this weather was great!
Good morning, Phil. 57 here, 100% chance of rain, might clear up in the afternoon and be in the mid-70s.
UNC-Charlotte has a lot of foreign students. I remember when Beau graduated from the business college, there were nearly 100 guys from India getting Masters in computer science. They probably all went to work for the banks and now play cricket on the weekends out here in the 'burbs.
Ohio state has a lot of international students as well. Trump's actions will reverberate across many universities across the country, almost all of which don't have Harvard's multibillion dollar endowments.
I know why Trump is doing it, but I disagree. I think the solution is to hire more people to vet all immigrant students, and deny those who are problematic. Err on the side of caution. Right now we don't.
With no small irony, a big chunk of my professional career (including academia) occurred because a Chinese student was denied a visa just as the fall semester started. The student had a graduate assistant position, and computing services didn't want it to go unfilled (use it or lose it). They called the business school; the qualifications were 'enrolled' and "has a pulse". I met those qualifications.
I don’t disagree with vetting. I disagree strongly with abruptly ending, even suspending, international student admissions across the board. This is Trump’s “bull in a china shop” approach, which is harmful even when I might otherwise agree.
My sentiments exactly. There are many ideas that he is pursuing that I favor, but am disgusted at his lousy means to the ends. Most of it will either come to naught or prove completely counterproductive—especially policies that get a bad rap due to incompetent and/or brutal implementation.
Very interesting comments, Jay. I appreciate your going into this in more detail. A phrase that jumped out at me was "Francesca Gino, a celebrated behavioral scientist".
Do you think the fact that Dr. Gino is a "celebrated" "scientist" is a significant factor in the decisions she made when the data nerds raised questions about her research?
What kind of study might be done to evaluate whether a person identifying as a "scientist" is more or less likely to treat accurate data collection as a matter of fiduciary responsibility, rather than in a more loosey-goosey, outcome-oriented sort of way? What about a person who is identified as "celebrated" or "celebrity" or otherwise outside the run-of-the-mill?
The discussion yesterday both here and at the Mothership reminded me of a podcast Jonah Goldberg did with a teacher/writer in the criminology field. The guest kept proudly self-identifying as a "scientist," to the extent that it really caught my attention as a listener. It seemed to me that he was more like a data researcher, which is a worthy occupation ... but is it the same as a SCIENTIST?
I think you might be right: she wanted to produce something 'Big" once she was at Harvard, and this research wasn't it. Professor Gino was graduated from the University of Triente; which is a pretty good university, but not at the Harvard level. For her to get selected within 6 years to jump from UNC to Harvard meant they thought she had "Star" potential. That's a pretty big jump up. At least two of her co-authors were "Stars" as well (Bazerman and Arielly). So she was expected to produce big results, field changing results. As I said, when I saw the effect sizes I was shocked.
This is a key point. Modest changes ought to produce modest effect sizes. Introducing a signature to a firm where there was none previously, that might produce a big change. But shifting it from top to bottom on the form? It felt modest to me.
I didn't say anything when I first saw it because it wasn't my field, so being critical would have gotten me labeled sexist, or a conspiracist nutter.
Scientist is a term that doesn't have real meaning. I am considered a "social scientist" because Management is considered a social science, albeit set in a professional school. I only use the term to set myself apart from physicists and the like. Data scientist is a very popular term for those who study how to gather and handle extremely large data sets. I don't like to use the term because it has "prestige" connotations.
Thanks. I agree that "scientist" is a term that is missing a firm definition. I brought that up yesterday at the Mothership, and some people thought I was making a point about policy or about people, rather than about what words actually mean.
There’s something in there related to the distinction between gaining prestige and asserting dominance. In the dominance hierarchy, the dominant ones, the kings of their own hills, demand that others approach them obsequiously.
Those who gain prestige are largely given it by popular acclaim. Prestige isn’t when the bearer demands fealty from others. In small-scale tribal societies, prestigious individuals, like the village elders, tend to be modest and self-deprecating, while they are widely acknowledged as especially skillful in specific areas of their expertise.
Dominance is asserted and requires the use of force or threat of violence. It is the ancient animalistic trait with roots going back to the simplest creatures, used to determine who gets the first, best access to scarce resources. The top of the academic pecking order looks a lot like a dominance hierarchy to me.
I get the distinction you're making between "dominance" and "prestige." That's from the book you've been reading!
I would suggest that there's a mix of the two in the academic situation. Maybe first a person acquires prestige, and then the person gets into a dominance hierarchy.
Also, I wonder if one of the elements here is Dr. Gino's being a woman. Did she think she would be protected from consequences because of that?
Just to clarify: Not sure exactly what I had in mind, but I think the dominance/prestige distinction was more relevant to the podcast you mentioned where the person had to keep saying “I’m a scientist.”
That caused flashbacks to the climate “debates” where you’ve had *some people* declare themselves not only “climate scientists”, but also arbiters of who else on earth gets to call him/herself one, too. Crazytown.
Interesting point. Do you think a person proclaiming himself to be a SCIENTIST is asserting dominance over the non-scientist person, such as a journalist? If I correctly understand the distinction, "prestige" is conferred on a person by others, rather than claimed and defended.
Sounds like a dominance play to me. “Don’t question what I say; you’re not qualified.”
That's exactly what it is. And it is used far too broadly.
I tell people here I am a professor, so they understand the context from where I am writing. I don't do it to gain extra credence as to what I write.
"Of course I am a social scientist! Why do you think my puns are so good"! I don't think that works, does it? 🤔
Maybe. Or maybe she was encouraged to think that way over her career.
I imagine it’s one of those things that happens to people who start off by bending the facts a bit and then get used to it, once they find out they can get away with the behavior. I’m sure it’s like the progression of Evil (to cite Baumeister’s book): the subject tells herself she’s doing important work for a good cause, and is sure the outcome is correct, but just needs a bit of assistance until the *right* study design becomes clear.
Rule-bending and -breaking are things we get habituated to, though, and once we’ve reduced our inhibitions the initial slips become an easy slide. The subject believes her significance and importance constitute a just cause in itself, and so on.
As with the distinctions between dominance and prestige, there are grey-zone boundaries where the two get hard to tell apart. Thus it is for a striving academic—fed on years’ worth of people with prestige telling them how important they are—to begin to see their own efforts as too important to let accurate data reporting get in the way, and so on.
Data Colada found four separate papers where there were issues. So once she began to fall, she fell all the way.
Interestingly enough, the first paper was one where the "big name" author was Dan Arielly (a big name) was the one being checked out. He claims he didn't modify the data, it was given to him as such, and he didn't bother to check to see if it had been modified. But she was a co-author. Did she learn from him, or was she cheating back then.
But the later papers all showed the same pattern: duplicate influential data points to improve a finding from no significance to significance.
"A renowned Harvard University professor was stripped of her tenure and fired after an investigation found she fabricated data on multiple studies focused on dishonesty.
The irony. It burns.
I've followed this story on multiple sites, substacks and pods. Its really quite incredible.
I followed it at "Data Colada", because I enjoy nerdy things. And how they found it was nerdy (and I mean that with affection).