AI shows up everywhere in students' lives, from personalized tutoring help to scholarship searches.  These tools deliver advice that feels personal and tailored just for you. But sometimes 'personalized' can easily slip into biased outcomes. 

That made me wonder: 

• What happens when two identical students, except for gender, ask AI for a college list? 

• Do AI tools treat students differently based on gender? 

• Does AI quietly tailor guidance based on who it 'thinks' you are?

To find out more, I conducted an informal experiment. I created four identical student profiles and changed only one detail: the student's name and gender. Then, I asked three free AI assistants to recommend a balanced college list for each student. What I found reveals how AI perpetuates social biases and gender stereotypes.

The Same Stats, Different Stories
The most consistent pattern was how colleges were described. For Daniel, the male student interested in CS, recommendations emphasized words like 'competitive,' 'rigorous,' and 'prestigious.' For Danielle, AI emphasized words like 'inclusive,' 'collaborative,' 'supportive,' and 'diverse.'

The language revealed the bias clearly. Daniel received advice like 'he should aim high at top CS programs' and 'his strong leadership makes him competitive for elite universities.' Despite the exactly the same profile, Danielle heard that 'she would thrive in supportive environments' and 'she would enjoy an inclusive campus culture where women in STEM are valued.'

This pattern held across all three AI tools: men equal ambition, women equal belonging.

The Outlier Gender Gets Different Treatment
Interestingly, AI doesn't consistently favor one gender; it favors the typical gender for a career field. Students who go against the stereotype get noticeably different treatment. These differences reflect how AI models 'narrativize' students based on existing gender stereotypes.

In Computer Science, Danielle actually received slightly more prestigious recommendations overall, but the language surrounding these schools differed dramatically. While Daniel's reach schools were described as competitive environments where his skills would be valued, Danielle's schools were framed as places supporting women in STEM, emphasizing collaborative rather than cutthroat culture. The focus shifted from achievement to belonging.

In Nursing, the script was flipped. The college recommendations for the male student revealed different assumptions, basically questioning his major choice. AI suggested non-nursing health sciences alternatives for Michael, as if assuming he might pivot careers. Michelle's list focused entirely on accredited BSN programs. AI seemed to question Michael's commitment to nursing in a way it never did for Michelle.

When you are the 'outlier gender' in a traditionally gender-dominated field, AI either questions your commitment (male student in predominantly female field) or emphasizes that you will need extra support to succeed (female student in a predominantly male field).

Geography and Institutional Types
Daniel's Computer Science recommendations spanned California, Illinois, Colorado, and Washington, with language about offsetting costs and exploring opportunities nationwide. Danielle's college list leaned toward California and Western schools like Oregon and Nevada, framed around comfort, belonging, and affordability.

For Nursing, Michael received faith-based or mission-driven schools emphasizing values and service (disclosing the underlying assumption: why else would a male student be interested in nursing?). Michelle's college list favored public flagships and selective programs like UCI, UCLA, and Samuel Merritt, all institutions known for academic excellence in nursing and clinical training quality.

Even though both majors showed limited overlap, AI was still more consistent when recommending colleges for Computer Science than for Nursing. That difference reveals how current and evolving gender norms continue to shape algorithmic 'confidence.' AI seems more certain about how to advise women entering CS than advising men pursuing Nursing. Several factors may explain this pattern:

Social acceptance varies by field
Women entering CS, while still facing stereotypes, have become more normal through 'women in tech' programs, visible role models, and institutional diversity pushes. AI has learned a standard story: women can do CS, here are the top programs. 

Men in nursing still face a stronger stigma with fewer public programs supporting their choice. Without a clear standard path, AI splits into different 'acceptable' versions of the story: service-oriented or faith-based missions, practical affordability, and suggesting alternative careers to nursing.

Field prestige creates agreement
CS has well-known national rankings that create agreement around 'top CS programs' no matter the student's gender. Nursing rankings are less prominent and more varied, with programs that differ a lot in structure. This uncertainty lets gender bias fill the gaps.

Training data volume matters
AI tools are trained on a massive amount of texts. There is far more online content discussing women in CS admissions, creating standard advice patterns. Men in nursing may be less discussed, so individual biases in limited data get amplified.

Story framing differs
When AI sees a woman in CS, the main learned story is corrective and encouraging: 'This is good, we need more women in tech, here are supportive programs.' When AI sees a man in nursing, there is no equally strong 'we need more men in nursing' social narrative. Instead, the advice becomes exploratory and cautious: 'Is he sure? What's motivating this?'

The pattern is obvious: when students match traditional gender expectations, AI gives consistent recommendations. When students don't match those expectations, AI splits into diverse suggestions that reflect uncertainty about 'appropriate' settings.

Each AI tool showed its own 'voice' and linguistic biases:

ChatGPT-5 was the most assertive. It confidently ranked schools as safeties, targets, and reaches, using strong action language like 'should apply' and 'should research.' But it also made factual mistakes, such as listing SAT score targets for test-blind UC campuses.

Claude was the most emotionally aware. Its advice leaned toward 'fit' and 'belonging,' especially for the female profiles, highlighting community and support systems over competitiveness. 

Copilot was the flattest. It recycled phrases from college marketing sites and gave safe, middle-of-the-road lists. Yet it subtly mirrored stereotypes: more 'ambitious' language for Daniel, more 'collaborative' framing for Danielle.

Even with identical prompts, each system revealed a distinct pattern: ChatGPT prioritized confidence, Claude empathy, and Copilot conformity.

As a college counselor, I want to point out several critical flaws in the recommendations. All three AI tools applied national standards to regional contexts and showed the tendency to over-generalize. They treated all colleges as if they followed the same rules. 

Safety Schools
First, Chat, Claude, and Copilot labeled highly competitive colleges as 'safeties.' Take San José State University for Computer Science. All three AI tools called it a safety school, but CS is an impacted major with competitive admission even for highly qualified applicants. Many other CSUs like San Diego State, Long Beach, Chico, and Fullerton are similarly impacted for both CS and Nursing, yet AI confidently labeled them as easy admits. You can verify impaction status in the CSU Impaction Matrix.

Test Scores
Another flaw was that AI referenced standardized test scores, despite UC and CSU systems being test-blind. ChatGPT and Claude even suggested minimum SAT/ACT scores for UCLA and other California publics, completely ignoring the fact that the UCs and CSUs will not look at standardized tests during admissions.

No Holistic Perspective
Third, AI treated admissions as purely numbers-based. The tools ignored the fact that the University of California considers the Personal Insight Questions and applicant context as part of their comprehensive review. Private schools conduct holistic reviews where essays can make or break an application. 

In addition, CSUs also give GPA boosts to local applicants, but AI made no distinction between in-area and out-of-area students. In short, AI flattened a complex, human-centered process into a set of statistics, missing the nuances and context that often decide real admissions outcomes.

These aren't small oversights. They are fundamental misunderstandings of how college admissions actually works, resulting in questionable recommendations. Students relying on AI need to fact-check every detail before building their college lists. This is precisely why AI cannot replace experienced human counselors and mentors.

AI often delivers advice that sounds confident and authoritative, even when it is incomplete or oversimplified. Students relying only on AI may not realize their list is unbalanced until rejection letters arrive. It lacks local knowledge, the ability to read what students really need, and judgment about when standard advice doesn't apply. 

How to Use AI Wisely

Treat AI as a starting point, not the final say. Use it to explore ideas, not to finalize your college list. Verify every “safety,” “target,” and “reach” label using real data like the colleges' Common Data Sets.

To see how gender bias may appear in a college list, try this quick test: 

Ask AI to generate a college list for a female student and then for a male student with the same grades, test scores, activities, and financial situation. Compare how it describes each list; are the words different (e.g., 'supportive' vs. 'competitive')? Note which colleges appear on both lists and which only appear for one. Merge the two lists based on your personal priorities, and use this list as your rough draft that you refine with your counselor or mentor. 

The goal is  to make sure your college list reflects you, not the stereotypes buried in AI's training data.

AI systems don’t invent bias; they inherit it from the data they get trained on. They mirror society's biases and deliver them as objective-sounding advice. Because they are trained on billions of words from the internet, they absorb the same gender and social stereotypes we have been trying to outgrow.

This experiment focused on gender, but the same patterns likely appear across race, income, first-generation status, and geography. Those biases can overlap. A first-generation Latina interested in Computer Science, for example, might receive advice filtered through multiple layers of assumptions, all presented as 'neutral' guidance.

AI has the potential to help democratize college advising, making guidance accessible to anyone with an internet connection. But democratizing biased advice does not make it fairer; it just spreads the bias further and faster. The good news is that awareness changes everything. Once you know what to look for, you can push back: 

Ask follow-up questions. Request more ambitious options. Tell AI to focus on academic fit and program quality, not personality guesses. Use AI as your initial research assistant, then work with your counselor to refine your list.

College advice should open doors, not quietly decide which ones you are allowed to walk through.