The Terror Queue: Google and YouTube moderators speak out


(dramatic music) – Part of the job description was, “You will be part of a team “that protects free speech online,” which makes it seem very heroic. It felt like you were putting
on a cape working at Google. – Over the past year, I’ve
been reporting on the lives of Facebook’s content
moderators in America and they’ve told me about their low pay, their dire working
conditions and in some cases, the long term mental health consequences of doing the work that they do. A content moderator is kind of like a police officer for the internet. If you ever see something that you think doesn’t belong on a
site and you report it, that report is gonna be
reviewed by a human being. While a lot of what they
see is really benign, like spam, for example, some
of it’s really disturbing. I’m talking about murder,
terrorism and child exploitation. Recently, I started
seeing out people who did this kind of work for Google and YouTube. I wanted to see how their
experiences compared to the ones I had heard about already. What I did learn surprised me. (dramatic music) – Part of doing our job
and how they would make us feel better about it was that, “You guys see this so other
people don’t have to see this.” (dramatic music) – Over the course of my reporting, I talked to both people who
worked at Google full time and people who had been hired on through third-party contractors. It became clear to me that
no matter who hired you, doing this job over a
long enough time period can cause significant
mental health consequences. But it also became clear
to me that there is a big difference in how
Google employees get treated and how those third-party
contractors get treated. Today, a former full-time Google employee named Daisy Soderberg-Rivken
is going on the record to talk about her experiences
as a content moderator. She had access to all the
perks and all the benefits that come with being a
full-time Google employee. But at the end of that
day, that didn’t save her from the consequences of doing the job. – I was a legal root removals associate, which is a very fancy way of saying I was a content moderator at Google. – Let’s talk about what
the job actually was. You show up, you have your orientation, you sit down at your computer,
it’s time to do your job. What is your job? – You usually start your work
by going through a queue. So you’re assigned to
a queue based on either an issue area or a geographic area. I focused on the French market, because my first languages
were French and English and I also worked on our child
sexual abuse imagery cases and our terrorism cases. – And you were working
primarily on web search, right? – Yes, we as in-house content moderators, we would usually handle more
high level, complex issues. Certain things that
were very high volumes, such as defamation and
copyright were typically sent over to contractors. They would then escalate
to us if it was kind of a gray area, but if it
was even a gray area for us, we would then escalate to our council. It was kind of levels of
how specialized we were. – At what point did you
start to feel like you were seeing more disturbing
stuff than you expected? – Very early on. They said we would be analyzing
child sexual abuse imagery but I remember clearly,
in parentheses, it said, this kind of content
would be limited to one to two hours per week, when in reality, we were understaffed,
so we would be in there sometimes five, six hours a week, which sounds like nothing,
but it’s actually… – Oh, it sounds like a lot.
– It’s a lot. – Yeah, yeah. When do you first notice
that doing this job was starting to affect your mental health? – When I was walking around
San Francisco, actually, and I was with one of
my friends and we saw a group of kids, toddlers, that were hanging on to one of those ropes so that they don’t go far. I looked at them and then,
I kind of blinked once, and suddenly, I just had a flash of images of some of the images I had
seen, children being tied up, children being raped, at that age. This is three, three years old. I kind of like stopped and
I was kind of blinking a lot and my friend had to make sure I was okay and I had to sit down for a second and I just exploded crying. She was like, “What just happened?” And I couldn’t explain
it to her and I just, these racing thoughts and
then, an instant panic attack. I was having nightmares,
I wasn’t sleeping, I had spent multiple days
just crying in the bathroom. I was having all of these panic attacks. My work productivity just dipped. Finally, my manager was like, “Listen, we really need you “to step up your productivity game.” I just snapped and I
turned to him and I said, “Do you understand what we’re looking at “and we’re not machines, we’re humans. “So we have emotions and those emotions “are deeply scarred by looking at children “being raped all the
time and people getting “their heads chopped off.” It was like there was no escape and yeah, I finally snapped and they took that as, oh, she needs to take a
second, she needs to breathe. And I was said, “No, I need to leave.” The free food, the nap
pods, all these benefits, this doesn’t mean anything if this is, if this is my day-to-day. – Daisy helped me understand
how hard this job is to do even when you work in the
greatest office in the world. But the truth is that
most people don’t work in an office half that nice. One of Google’s biggest
projects that it has to moderate, of course, is YouTube. When it comes to YouTube, Google has decided to
give most of the work of content moderation to
third-party contractors. Recently, I went to Austin,
Texas, to meet with a group of moderators who work for
Accenture on the YouTube project. Specifically, they work on
what is called the VE queue. VE standing for violent extremism. 120 times a day, they review YouTube videos
that have terrorism, graphic violence and
other disturbing content. You’re about to hear from one of them and we’ve altered the audio
to protect their identity. – [Moderator] So, at the beginning, they told you to watch some videos. You’re going to take some actions. You will apply the YouTube polices. But you don’t feel how this
is going to impact you. – In some ways, the content
moderators who do this work for Google and YouTube are treated better than the ones who work for Facebook. Most prominently, they get two
hours of break time each day. Basically, two hours of
paid leave in which they can recover from the
challenges of doing this work. But, most of them aren’t able to take a full two hours a day. – [Moderator] They’re forcing
you, micromanaging you to have to be sitting on the
desk five hours and a half. And if you don’t, there is
going to be a punishments. The schedules will be changed. You will be on night shift. And this is going to
affect my wellness time. I will never take my three hours. (dramatic music) – [Casey] What kind of things do they do that make life hard? – [Moderator] They always have
complaints about everyone. You know, like, I have something on you. If you make any problems, you know what? This is the reason that I can fire you. – [Casey] Right, right. – [Moderator] One of the
things that they always saying is if we miss one agent
tomorrow, we get another 10. – [Casey] So they’re
constantly reminding you how easily you can be replaced? – [Moderator] Yes. The problem that’s they
feel stuck somewhere. They can’t leave the work because they have responsibilities. He have bills right now he have to pay. – [Casey] So it sounds like
people feel kind of trapped. – [Moderator] They are. Yeah, that’s a good word. – When I brought all this to Google, the company told me
that it takes the health of its workers very
seriously and pointed out that it offers onsite counseling to both its full-time employees
and to its contractors. I think it’s worth pointing out, though, that even though Daisy had
access to that onsite counseling, the counselor she had ultimately told her to go seek outside help
and get a therapist. Daisy also eventually took medical leave and ultimately got an emotional
support animal to help her. It’s a dog named Stella. – Hi five. Found a psychiatrist
and I found a therapist. The psychiatrist put
me on antidepressants. I was diagnosed with
chronic anxiety and PTSD. And then, I started seeing
a therapist just to talk through these things and she said, “Is legal removals
associate anything close “to a content moderator?” And I said, “It is a content moderator.” And she said, “Trust me
when I say you are not “the first person that I’ve seen “with this particular issue.” – It seems like recovering
from doing this job has itself been a full time job. – Oh, yeah. Whenever someone talks to
me about content moderation, I say, “I’m a recovering
content moderator.” They’re like, “Oh, you
talk about it like it’s, “it’s like alcoholism.” And I said, “Well, you
never fully recover.” – One of the things that
is so interesting to me about your story is that you
are one of the very few people I’ve talked to who did
content moderation as a full time employee of a
company, rather than a contractor. You had access to six months
of paid medical leave. A contractor who’s moderating
for YouTube in Austin doesn’t have that same access. – I had those months to
think about my choices and to think about ways out without having to deal with unemployment
or having to deal with how am I gonna pay rent. I know those contractors
don’t have that opportunity. – The contractors I’ve
talked with in Austin ,are making $18.50 an
hour, about $37,000 a year. Does that seem like a high
enough wage given some of the risks involved? – Absolutely not. There’s never gonna be enough
money to make this okay. I’m gonna be clear about that. But, I think that you need to
pay contractors proportional to what they’re going through,
the impact of their work, because this is so vital to the business. – Let’s put a fine point on it. If Google can’t exist without
the work that you did, right, like you were responding
to official legal requests from governments–
– Yup. – That would have
otherwise shut Google down in their country–
– Yup. – If you didn’t respond. – Exactly. – So this is very high stakes work. And yet, for some reason, these companies have just
chosen not to value it. – Yeah, I think that
contractors are so essential, especially considering
how much volume we have. We need as many people as
we can doing this work. We also need to change the overall system and the overall structure of
how this work is being done, how we support these people,
how we give them tools and resources to deal with these things. Or else, these problems
are only gonna get worse. (dramatic music) – Hey, thanks for watching
and don’t leaven any comments. It will only create more work
for the content moderators. But, if you want to know
more about what’s going on with our wayward tech platforms,
I write a daily newletter called, The Interface,
about the collision between big social networks and democracy. You can find it and subscribe for free at theverge.com/interface. (dramatic music)

100 Comments

  1. toddler's ! ! ! 3 yr olds getting raped all the time ! ! ! that's 100% justifiable for a global nuclear apocalyptic cleansing RIGHT ? 🇬🇺👊🏾

  2. 9:47 – It's more like soldier.
    11:43 – The sheer volume of horrible things is what's terrifying. That there exists that much horrendous evil in the world is truly frightening. :- Humans need extinction in the worst way. ¬_¬

  3. Thr thing thst pisss me off about this is the low pay, goggle and facebook are some of thr richest companies on earth!! They have no excuse not to over pay these people for the horro they have to go through

  4. Big eye opener for me. That’s pretty sad. I hope things get way better for her and everyone else that has to do this. Thanks for enlightening all of us.

  5. I am sad for the immigrants. Maybe they escaped the horrors in their own countries, but got caught in new horrors in the host countries.

  6. This is definitely an area that needs improvement in working conditions but I'm really not sure how. AI is coming but not that it can take away this work from humans especially those Grey areas will always be dealt by humans.

    But she's awesome to share it publicly. Kudos to her confidence.

    And also to Verge for bringing this out and not only being a partner to tech cos like many other channels.

  7. Lets not forget the bigger picture here that there are many disgusting human beings around the world who should not exist.

  8. I'd like to think that I'm strong enough to moderate violent videos towards kids and animals, and just people in general, but the truth is it scares me — that I couldn't un-see the things moderators see. I don't think anyone is strong enough to do that type of work.

  9. These big companies that contract work to outside ones are everywhere. Amazon, UPS, Fedex, USPS, Youtube, Facebook, Chase Bank, Wells Fargo, etc.

    Anyways, I wouldn't mind being a content moderator, at least as a part time gig.

  10. this explains why there is a bit of a bias in moderators. There was a study that showed the feeling of disgust is not felt as strongly, or at all, by people who vote Democrat. So eventually moderators would in the majority end up Dem voters.

    such a crazy hard job for anyone with empathy.

  11. Does Google have anything to say about those cases? I truly believe in them, but I also want to hear what they have to say…

  12. It’s very interesting to learn about this and I value anyone who does this for a living. Whether you moderate for any kind of community on the internet. Thank you.

  13. I honestly already have things that are hard to get out of my head, and that’s just from listening to their stories. I’d really rather live on the streets than do that job! The only ones qualified in my opinion are psychopaths. Since their empathy is missing entirely (not their fault), so it shouldn’t haunt them…

  14. Youtube recommended me to watch this video and now I want to work as a Legal removals associate.

    I think YouTube knows me better than ————anybody else on earth 🌐

  15. "There's never gonna be enough money to make this okay." – Too bad. The American legal system disagrees, collect $200 when you pass go.

  16. Maybe it's just me but I find the way Casey sits during the interview with Daisy disrespectful – or lack of a better word. As a journalist myself, I can't imagine ever "relaxing into"an interview in that manner. Lean in and truly show you're listening is how I'd do it. Unless of course, this is what helped her relax into that conversation.

  17. This is a very good video, I'm not used to seeing this sort of investigative journalism from The Verge (I don't mean that in a bad way, it's just that The Verge usually does reviews, which I've been watching for years). But I think this investigative stuff is really great, you should make more of it.

  18. Tech industry is the modern age monster , its the enemy of all countries , its the ak47 who any one can rent , the number 1 public and humen enemy as i c it since years are both Facebook and google . As i understand only some EU countries like Germany are standing against those monsters by regulations am not just taking about moderators its about the whole structure of these companies , google and facebook violate our privacy in every sec more than million times, unstoppable data collection , storing data on servers in international seas , what did google offer in 20 years ?!!!! google map which work by spying on ur phone gps and thats how it tell you about traffic jam thats one example , every thing google offer is by spying on ppl and learn human behavior , fb sale our data private data for third party dev , fb are accused by FBI they manipulate USA election , what MORE , those two beasts should be stopped by governments.

  19. These are the same jerks that suppress Infowars, ufo videos, conspiracy videos, basically anything that takes you down the rabit hole.

  20. What I don't understand is why there are no blurry filters on those videos? If it seems like a inappropriate video, they can decline it, without seeing all the details in the video. If they are not sure, they can slowly reduce the amount of blur, until they are certain the video is inappropriate.

  21. I remember a time when YouTube was full of fun, entertainment and awesome science stuff from all over the world. Now I think its more of 'we upload whatever we want' kinda videos

  22. Sorry, I can't see how an online moderator would have a tougher job than the emergency services. IRL, full sights and smells.

    It's not about psychopath or not. It's not about being weak or not. It's resilience and desensitization. Ordinary people can easily work years in the emergency services without side effects like PTSD or nightmares.

    At the same time, there will be some who can't handle it, that's fine too.

  23. With the face recognition tech available, maybe they could at least automatically track and blur faces. I imagine that it could help reduce traumatic mental effects.

  24. Actually, now that I think about it. A psychopath would be really good at this job with little ill effect since they have issues with empathy. I wonder what would happen if personality tests were issued to make sure a certain level of Psychopathy existed before being hired.

  25. I believe I know how she feels. And I also believe this is finally an opportunity to employ and assist self-loathing paedophiles. If they can form a tier of reviewing both potential child abuse along with generally violent videos it will surely help them become lower risk.

  26. I like Casey's reporting, but at the end of the day it's a little meaningless because it doesn't propose any solutions. At least in the Facebook case, there were actual things that were very poorly handled. But here, there isn't much more that can be done here. Maybe some pre-screening to hire people who are more psychologically resilient.

  27. Can you imagine 900 dollars per hour vs 15 dollars for 8 hours daily of an employee of a contractor in the Philippines at UpTown Mall, Bonifacio Taguig City for the same work? And no counseling or therapy nor sick leaves. Still, people in a developing country would risk their mental health because no other jobs here would amount to a near living wage, that's why they choose to work for a vendor for Google/ YouTube.
    I personally got axed last week from another account at Google because my productivity went below expectations. During my brief stay I had to bring myself to the emergency room of a nearby hospital when my blood sugar went up to 361 and they won't call an ambulance. They kicked me out finally when I got slipped disks (c4-c6) in my vertebrae and I had to go to physical therapy three times a week. That is when my productivity went down because I could barely move the mouse with my right hand from the pain and numbness. After Thanksgiving my supervisor told me I failed November's numbers ( ad sales North America ) and just one week after finishing the one month PT sessions. Yeah , this is a great way to greet me a happy holiday. Jobless and uncertain for 2020. Thank you Google.

  28. With Picture and Video Processing advancing these days and AI playing a major role, is there no way for Google to use AI and Machine Learning to use it as a tool in place of these content moderators? That can be more efficient.

  29. This was incredible. Just like Johnathan Morrison's Mac Pro review, this video is the kind of content that is raising the bar for what YouTube could be. It is so valuable to be able to shed light on some of the inner-workings of seemingly "perfect" work places. This just leaves me wondering if there is anything we could do to help.

  30. This is one job that would hugely benefit from robust AI to remove the most graphic of visual content, which no-one should have to see apart from specialist law enforcement. I'm surprised this doesn't happen already.

  31. Screw Joogle & JooTube they are politically biased & pushing ideological subversion thru censorship, anti-freespeech, & anti-gun.

  32. This makes me happy that these people are getting blackpilled. Maybe they'll question what they do. Unfortunately, those who tell them to institute censorship aren't the ones feeling discomfort.

  33. I liked it more if the reporter didn't take side and let the story to be told by itself, regardless very interesting story.

  34. From someone who is not doing the exact same thing, in the Netherlands, but who can relate to it: this is some good reporting. Good job 👍

  35. Could we train AI with millions of videos to take care of a fraction of the videos? It won't do everything, just lighten the load.

  36. I think Google as a leader in Maschine Learning should offload a lot of this to algorithms. Image recognition is very reliable these days.

Leave a Reply

Your email address will not be published. Required fields are marked *