Deepfakes and Identity-- Problems, solutions, focus on technology
Deepfakes & Identity: Problems, Solutions, Focus on Technology
Session: 10H
Convener: Kathryn Harrison
Notes-taker(s): Sankarashan
Tags for the session - technology discussed/ideas considered:
Discussion notes, key understandings, outstanding questions, observations, and, if appropriate to this discussion: action items, next steps:
Evolution of the work presented at the last IIW - 6 months of work and research since then
Deepfakes as got a lot of attention but it is part of a much broader umbrella of disinformation. 96% of deepfakes are deepfakes porn (largely celebrities)
Categories of information
- information which is false → misinformation (information which is incorrect, out of date, context)
- True information but used to harm → malinformation (doxxing)
Disinformation exists in the intersection of the above. Is disinformation propaganda - Yes, generally. Depends on the content of the propaganda (but can exist in either category). Disinformation is one technique in a broader set of PsyOps. In modern information age era - disinformation is targeted to everyone (eg. deepfakes, bot attacks etc)
Cheapfakes (eg. Nancy Pelosi video : photo editing, miscontextualizing) Deepfakes - generative adversarial networks (GAN) - uses AI [Generator | Discriminator] Whose images are being used? Is there consent? ← first set of issues around identity as they arise Lipsync video; deepfake audio ; real time re-enactment
- Liar’s dividend - that this technology exists - it is easy for anyone to introduce doubt in an image/message. Creates a fundamental doubt around trust - what is real/not real
- Market manipulation via fake tweet (S&P movement of 26%). Other examples being French transport companies
- Social engineering - whole new set of technical tools
- Extortion and Harassment - case of Rana Ayyub (deepfake porn circulated extensively on social media)
Coalition of stakeholders across aspects needed to address the challenges
- Technology & Policy
- Education - attention economy and deficit causing unaware nature of impact (cognitive laziness). COVID-19 has been an exceptional test bed - large data gaps causing a lot of interest in addressing that
How do you create a vetting method? Consider the lifecycle of information:
Maker → Creation → Distribution → Believability → Impact
To what extent is identity tied to the production/creation of content?
Algorithmic propaganda = mass scale of distribution (technology and policy intervention to gate)
Believability - (1) confirmation bias (2) cognitive laziness.
‘One Single Solution’ does not exist. Rather an integration approach is expected to work. Detection techniques help identify what is fake. How do you verify the authenticity of a content? How do you help a much broader audience understand what is happening in the space?
Watermark/DID into specific images would allow tracking the images under review for authenticity. Would where it has been detected (uses cases of weaponized media) and similar point of information be recorded? “information virus vectors”
- identifying bots/non-humans
- identify what humans are being targeted through (what kind of data around the targets would need to be recorded) this deepfake
levels of combating information operations
- technology platforms and technology developers
- individuals
- society (herd/mass effect) - digital interventions/digital “de|re-programming”
Reverse image search is one technique that can help identify whether images are fakes.
“Digital integrity” - as an output result of the image under review. Manual process → automated flows to make workflows such as KYC etc to be more efficient.
What are the possible next actions from the deduction?
As a group the potential areas of collaboration
- standardization of explanations (taxonomy, Digital Harms Dictionary)
- designing a common understanding of what the solutions and interventions would look like
- marketplace for forensic capabilities into commercial applications
Cross platform incident reporting and log (3rd party auditing) - questions around access and roles would need to be worked out
Casenet - Missouri : traction - well vetted repository of facts through jurisprudence
“false hope” - Self reporting at companies that do not often get taken care of. The tool could mitigate that if the reporting does go to the companies to encourage action to be taken. Work being undertaken to understand the methods on which the inference can be translated into action - the scale and systematic process needs the right set of stakeholders. Privacy and usage policy would need to be strengthened as well. Working on items that could break the law providing guidance on how the reporter could report those that CAN take action. Connecting legal services to local law enforcement. citation/reference of josh_emerson (Twitter) for methodology about identification of bots/fake accounts
People sharing information from fake accounts
http://www.youtube.com/watch?v=4fYZpRBuh-s&feature=youtu.be - to be able to intervene and defang prior to the issue becoming an incident
http://stoponlinevaw.com/media/press-release-report-on-stop-digital-voter-suppression/ http://stoponlinevaw.com/media/report-facebook-ads-that-targeted-voters-centered-on-black-american-culture/
Introducing a concept of peer review on analysis and inference being created on content being processed by the system. Or, a “reputation framework”
Measuring an extent of editing and manipulated in the content - giving people tools and capabilities to make decisions on that.
http://github.com/alievk/avatarify
What datasets are available for training? (as an anecdote - http://thispersondoesnotexist.com/ did not lead to an image of (woman of color) PoC - more diversity is needed)
