How To Build Context-Aware Systems to Avoid Context Breaches in a World of Intelligent Agents, IOT and AI

From IIW

How to Build Context-Aware Systems to Avoid Context Breaches in a World of Intelligent Agents, IOT, and AI


Day/Session:Wednesday 4I

Convener:Jeffrey Friedberg

Notes-taker(s): Jeffrey Friedberg


Tags for the session:

  • Contextual Integrity
  • Context-Aware Systems
  • Context Breach
  • Intelligent Agents
  • IOT
  • AI


Technology discussed/ideas considered:

  • Controlling Identity Attributes
  • Concept of “Contextual Integrity”
  • Examples of “Context Breaches”
  • Unique problems with Intelligent Agents and IOT
  • How to enable contextual integrity in systems


Discussion notes, key understandings, outstanding questions, observations, and, if appropriate to this discussion: action items, next steps:


See Link to Slide Deck for Details:

Controlling identity attributes

  • Many attributes describe us
  • Humans expect to be able to control who knows what
  • Sometimes easy (wear a shirt to hide tattoos)
  • Very hard in today’s cloud-based world
  • Using separate IDs (decentralized or otherwise) is not enough
  • Smart data scientists linking across Device IDs, IP address, other data …
  • By design, goal of many systems is to build a unified view of you
    • One big lump of identity attributes

Contextual Integrity

  • A key concept introduced by Helen Nissenbaum could help

27W4I.png

  • Privacy is defined as:
    • Appropriate flows of data based on context and expected norms
  • Example:
    • Doctor / Party
      • You visit doctor who is also a friend
      • Doctor treats a sore in a sensitive spot
      • You invite the doctor to a party
      • At a party, when you are standing with others, you do not expect the doctor to ask about the sore
    • Salary
      • In the US you typically do not share what you are paid
      • Other places, may be a norm (e.g., must report to IRS)


Context Breaches

  • Definition:
    • 'Data appropriate for one context is used in other that causes a range of harms '
  • Examples:
    • Itch cream
      • You are home and need to find some itch cream for your sore
      • You log in with your personal ID and search for itch cream
      • Next day you need to give a presentation to your boss’s boss
      • You log in with your organizational ID and need to surf the web
      • What are the chances you will see itch cream ads down the side of the browser?
    • Target
      • Company was able to deduce from purchases a young woman was pregnant
      • Young woman did not know she was pregnant
      • Congratulations were sent with discounts to account holder – her father!
      • Big surprise with big social/family consequences
    • Message leak
      • My colleagues were giving a big presentation to my boss’s boss
      • My manager was unhappy with the pace
      • He sent a nasty instant message to get on with it
      • On the big screen, a toast popped up with the message for all too see!
      • System was not context aware. Could have easily checked a presentation was active and the message was not to the same list of people as was invited to the meeting!
    • Cell Phone Wipe
      • I’m allowed to get my corporate email on my personal cell phone.
      • My young son picked up my phone that was locked with a PIN. Thought it was safe for him to play with it. 2 minutes later, he handed it back and phone was completely wiped. Lost 3 months of family pictures.
      • Company was worried I might lose my phone, a bad guy might find it, try to guess my PIN, and steal corporate secrets. They put a policy on the phone that would wipe all contents after 5 attempts to enter my PIN.
      • The issue isn’t my company wanting to protect their assets. The issue is the deletion feature was not context-aware. i.e. It’s OK to delete pictures of whiteboards at work. It’s not OK to delete pictures of my kids at home.
      • With pictures getting tagged with time of day and GPS location, and with facial recognition, the system could have distinguished my personal photos from ones in scope for deletion.
      • Side note: to protect against a child deleting all photos on a locked phone, all the company had to do was to set the maximum attempts to 10 instead of 5. On the iPhone, after the 5th try an increasing time delay is added and a child would quickly get bored and give up.
  • Range of harms for a content breach
    • From temporary embarrassment to life changing impact (e.g., leaking your sexual preference in some parts of the world could get you killed).
  • Need to consider context in design!
    • Most systems aren’t tracking context yet or using it to gate data operations and flows!
    • Should add “Context Breaches” to the threat model and test for it as part of the software development lifecycle!


Unique Challenges

  • Intelligent agents
    • Can be more effective if they know more
    • Higher levels of trust needed
    • Need to get context right!
  • IOT
    • Limited opportunity for consent
      • At a high-end store where I live, as you walk in, behind the fern, there is sign that says my location is being tracked (via my cell phone’s IEMI #) so they check my dwell time in front of different products and use that to market to me.
    • Must pass on context to avoid harm


Open Discussion

  • How do we enable contextual integrity in systems?
    • What goes in plumbing? In the UI?
  • Which context attributes to capture?
    • What to capture first? Work/School/Home?
  • How to model expected normative behavior?
    • Needs to be culturally localized
  • What tools can we leverage?
    • AI, D* Identity, Privacy Chain, …
Retrieved from "https://iiw.idcommons.net/index.php?title=How_To_Build_Context-Aware_Systems_to_Avoid_Context_Breaches_in_a_World_of_Intelligent_Agents,_IOT_and_AI&oldid=21915"