When called down to the principal’s office last October, high school student Francesca Mani learned that someone had taken online pictures of her and used artificial intelligence to generate fake nudes that were then shared on social media.
Thank you for reading this post, don't forget to subscribe!The teen was understandably shell-shocked, yet as she made her way through the halls and saw the clusters of crying fellow classmates also affected (while others laughed), anger and a sense of purpose replaced her own tears.
“I came home to tell my mom … ‘We need to do something about this,’ because I didn’t think it was fair,” said the Grade 10 student from New Jersey. “I wanted to protect other people, so they weren’t in the same position as I was.”
Since the incident came to light, the sophomore and her mother, Dorota Mani, have been meeting with U.S. state and federal politicians to press for updates to laws, school policies and codes of conduct to address AI being used for harassment and abuse.
Experts say it’s a growing problem, and Francesca said she hasn’t ever seen AI discussed in school assemblies about bullying or cybersafety.
“I’ve been fighting for this. My daughter has been advocating and pushing me to advocate for this because we do believe this is extremely important,” Dorota said.
“I think we need to educate our children … in a meaningful way that will resonate,” she added. “They’re young, you know, their hormones are raging… But then there’s also right and wrong, legal and illegal, and that should be clearly stated [in a way] that they understand.”
As similar incidents — including at a Winnipeg school in December — crop up, some experts are urging education officials to update and improve both curriculum and school policies to address technology-facilitated sexual violence and to better support teachers to tackle these topics in their classrooms.
Not simply ‘cyberbullying’
Though a relatively new term, tech-facilitated sexual violence covers a wide range of incidents increasingly affecting young people today, said sociologist Kaitlynn Mendes, an associate professor at Western University in London, Ont., and Canada Research Chair in Inequality and Gender.
What falls under this umbrella? Examples include: cyberflashing (receiving unwanted sexualized images), sextortion (coercion to share intimate images or videos), nonconsensual sharing of intimate images and cyberstalking via GPS locators in social media apps. AI-generated deepfake images and videos would also fit, Mendes said.
WATCH | Calls for better education:
Youth need better education on the risks of online sexual harm, report says
Tech-facilitated sexual violence and harassment are on the rise in Canada and a new report suggests that schools could be doing more to educated young people about the risks.
“Digital technologies are increasingly playing a role in the way that sexual and gender-based violence takes place,” said Mendes, co-author of a new report that digs into how school policies and curriculum in each province or territory currently address the topic.
The terminology is important because “if you have your intimate images exposed or shared without your consent, that’s not the same thing as bullying. And we need to actually re-frame those actions as a form of sexual violence because that also changes the ways that people respond.”
Mendes and her colleagues found inconsistencies in what students are taught about topics linked to sexual or gender-based violence. Depending on the region, these issues could be explored in a life skills class, phys ed and health, communications or perhaps even computer studies.
The report also noted that few jurisdictions are referencing more recent technology, like social media or AI, nor do most address the connections between what happens to students online and offline.
For instance, “they’ll talk about sexual violence or gender-based violence that happens offline without recognizing the ways that digital technologies are actually being used to kind of facilitate them,” Mendes explained.
“The provinces and territories do talk about cyberbullying, but they’re not even [touching on] … all the new developments that we’ve seen in recent years.”
WATCH | Consent and privacy in the classroom:
Teaching kids about consent, privacy as ‘building blocks’ before more complex talks with teens
Kaitlynn Mendes, an associate professor at Western University, outlines how teachers can start introducing principles of consent, privacy and online rights in younger grades, to prepare for more complex conversations when students get older.
There’s room for improvement across every province and territory, she said, including by starting students’ learning earlier. Mendes feels elementary-age children can learn about principles like respect, consent, right to privacy, bodily autonomy and being a good digital citizen, for example, long before they start learning about sexuality or sexual violence.
How educators frame their lessons is also important.
“Make sure that young people, when they’re using digital technologies, know their rights and responsibilities. They know how to behave ethically, but also they know what to do if or when things go wrong. They know who they can talk to,” Mendes said.
“We want to get rid of the shame, because I can tell you [after] talking to lots of teenagers, they rarely go to adults when things go wrong because they’re scared of being told ‘You’re an idiot. This is your fault. Your life is over.'”
More training, resources needed
Having spent nearly a decade delivering in-class sessions for high schoolers, professional development for teachers as well info sessions for parents, sexual health educator Carlie McPhee agrees that adults need to stay current about what’s going on in teens’ lives and remain open-minded so students feel safe in asking for help or guidance.
WATCH | Be non-judgemental, says sexual health educator:
Take time to learn, be non-judgemental with teens, says sexual health educator
Sexual health educator Carlie McPhee encourages adults to stay current with where teens are at and keep an ‘open door of communication’ to support them
The Vancouver-based educator has no qualms querying teens about what’s cool or what social media they’re using — “and they’re happy to let me know when I’m outdated” — since it offers context about their lives and opens the door for the teens to ask questions (queries about AI deepfakes, incidentally, began turning up late last fall).
Having both adults and friends their age speak openly “creates a support system… if something does come up that’s upsetting or if they’re being victimized in some way online by a stranger or someone in their real life community,” she said.
After hearing similar challenges expressed by many educators, one of McPhee’s goals is developing a slate of accessible resources teachers can use. They’ve told her they had little to no training in the subject matter, felt uncertain about what’s age-appropriate, worried about how to interpret vague curriculum expectations or are apprehensive about missteps — and any of these can impact what kind of education students receive.
“It’s tricky territory, without any professional development, to come in and manage different beliefs and value systems, different identities in your classroom in a respectful, welcoming, safe way,” McPhee acknowledged.
Still, as a sexual health expert teaching alongside classroom educator peers, she said, “we are working hard to be on top of this and make sure that we have that key messaging and information and skills to help kids stay safe, make informed decisions and know how to respond if something does happen. And also to understand their responsibility of how they need to be behaving online as well.”
