Page tree
Skip to end of metadata
Go to start of metadata

Overview

Below is an overview of the current discussion topics within the Trusted AI Committee. Further updates will follow as the committee work develops. 

  • Focus of the committee is on policies, guidelines, tooling and use cases by industry

  • Survey and contact current open source Trusted AI related projects to join LF AI efforts 

  • Create a badging or certification process for open source projects that meet the Trusted AI policies/guidelines defined by LF AI

  • Create a document that describes the basic concepts and definitions in relation to Trusted AI and also aims to standardize the vocabulary/terminology

Mail List

Please self subscribe to the mail list here at https://lists.lfai.foundation/g/trustedai-committee

Or email trustedai-committee@lists.lfai.foundation for more information. 

Participants

Initial Organizations Participating: AT&T, Amdocs, Ericsson, IBM, Orange, TechM, Tencent

Committee Chairs

NameRegionOrganizationEmail AddressLF ID
Animesh SinghNorth AmericaIBMsinghan@us.ibm.com
Souad OualiEuropeOrangesouad.ouali@orange.com
Jeff CaoAsiaTencentjeffcao@tencent.com

Committee Participants
NameOrganization Email AddressLF ID
Ofer HermoniAmdocs oferher@gmail.com
Mazin GilbertATT mazin@research.att.com 
Alka RoyATT AR6705@att.com 
Mikael Anneroth Ericssonmikael.anneroth@ericsson.com 
Jim SpohrerIBM spohrer@us.ibm.comspohrer
Maureen McElaneyIBM mmcelaney@us.ibm.com
Susan MalaikaIBM malaika@us.ibm.com
Romeo KienzlerIBMromeo.kienzler@ch.ibm.com
Francois JezequelOrangefrancois.jezequel@orange.com 
Nat SubramanianTech Mahindra Natarajan.Subramanian@Techmahindra.com
 Han XiaoTencent hanhxiao@tencent.com

Assets

- All the assets being 

Sub Categories

- Fairness: Methods to detect and mitigate bias in datasets and models, including bias against known protected populations

- Robustness: Methods to detect alterations/tampering with datasets and models, including alterations from known adversarial attacks

- Explainability: Methods to enhance understandability/interpretability by persona/roles in process of AI model outcomes/decision recommendations, including ranking and debating results/decision options

- Lineage: Methods to ensure provenance of datasets and AI models, including reproducibility of generated datasets and AI models

Projects

Meetings

How to Join: Visit the Trusted AI Committee Group Calendar to self subscribe to meetings.

Or email trustedai-committee@lists.lfai.foundation for more information. 

Meeting Content (minutes / recording / slides / other):

Date

Agenda/Minutes

Proposed Agenda:

  • Jim to get feedback from LFAI Board meeting
  • Romeo to demo AIF360 -Nifi Integration + feedback from his talk at OSS Lyon
  • Alka to present AT&T Working doc
  • Discuss holiday week meeting potential conflicts (28 Nov - US Holiday, 26 Dec - Day after Christmas)

Attendees:
Ofer, Alka, Francois, Nat, Han, Animesh, Jim, Maureen, Susan, Alejandro

Summary

  • Animesh walked through the draft slides (to be presented in Lyon to LFAI governing board about TAIC)
  • Discussion of changes to make
  • Discussion of members, processes, and schedules

Detail

  • Jim will put slides in Google Doc and share with all participants
  • Susan is exploring a slack channel for communications
  • Trust and Responsibility, Color, Icons to add Amdocs, Alejandro's Institute
  • Next call Cancelled (31 October) as many committee members will be at OSS EU and TensorFlow World

Notes from the call:
https://docs.google.com/document/d/1_PKbqRMQgHp1ieqooAheKSy3aYXnpHKTxLL5Ru3aHPw/edit

Attendees:

Animesh Singh (IBM), Maureen McElaney (IBM), Han Xiao (Tencent), Alejandro Saucedo, Mikael Anneroth (Eriksson), Ofer Hermoni (Amdocs)

Animesh will check with Souad Ouali to ensure Orange wants to lead the Principles working group and host regular meetings. Committee members on the call were not included in the email chains that occurred so we need to confirm who is in charge and how communication will occur.

The Technical working group has made progress but nothing concrete to report.

A possible third working group could form around AI Standards.

Notes from the call:

https://docs.google.com/document/d/18BejXILRRqXFRmBxb11tf4Bkhf9qiiVgfHlIL3CQQyM/edit?usp=sharing

 

Attendees:
Ibrahim. H, Nat .S, Animesh.S, Alka.R, Jim.S, Francios. J, Jeff. C, Maureen. M, Mikael. A, Ofer. H, Romeo.K
  • Goals defined for the meeting:
  • Working Group Names and Leads have been confirmed:

    Principles, lead: Souad Ouali (Orange France) with members from Orange, AT&T, Tech Mahindra, Tencent, IBM, Ericsson, Amdocs.

    Technical, lead: Romeo Kienzler (IBM Switzerland) with members from IBM, AT&T, Tech Mahindra, Tencent, Ericsson, Amdocs, Orange.

    Working groups will have a weekly meeting to make progress.  First read out to LF AI governing board will be Oct 31 in Lyon France.

    The Principles team will study the existing material from companies, governments, and professional associations (IEEE), and some up with set that can be shared with the technical team for feedback as a first step.  We need to identify and compile the existing materials.

    The Technical team is working on Acuomos+Angel+AIF360 integration demonstration.


    3. Possible Discussion about third working group

  • Discussion about LFAI day in Paris

  • More next steps
    • Will begin recording meetings in future calls.

Notes from call: https://docs.google.com/document/d/1QJ3t0YD3mzOa-gajjbHtgjXEO7_LPY1gmRW1SHcoNo4/edit

  • No labels

5 Comments

  1. Here are some additional notes taken by Jim Spohrer (IBM) <spohrer@us.ibm.com> - https://docs.google.com/document/d/1QJ3t0YD3mzOa-gajjbHtgjXEO7_LPY1gmRW1SHcoNo4/edit

  2. This document: https://docs.google.com/document/d/1QJ3t0YD3mzOa-gajjbHtgjXEO7_LPY1gmRW1SHcoNo4/edit?usp=sharing

    LF AI - Trusted AI Committee Call - Every other week
    (may change)

    Ibrahim Haddad (LF): I will set up "Trusted AI" Zoom to record future calls
    Ibrahim: Attendees should go to Wiki for agenda: https://wiki.lfai.foundation/display/DL/LF+AI+Trusted+AI+Committee
    
Jim Spohrer (IBM): We should rename ourselves when we join zoom - so people can easily see your company - I just did that (a good point to add to meeting start when we review the agenda)

    
Nat Subramanian (Tech Mahindra): Wiki todo: Need a section of wiki to highlight guest speakers

    
Ibrahim: Zoom todo: will set up a zoom for Trusted AI - and in future co-chairs will be able to record

    Animesh Singh (IBM): Showed a Google Doc with agenda - need to get link
    Animesh: Two working groups
 (WG) will have lead, core members, optional members and meet separately
    Animesh: WG Principles and WG Technical
    
Animesh: WG Techincal focus - landscape projects in Trusted AI area, and specifically those that are coming into LF AI to support Trusted AI work
    Animesh: Which OS projects will come into LF AI and which are being integrated into LF AI projects
    
Animesh: Face-to-face meetings
 for WG will need to be planned by WG leads
    
Animesh: WG should have outcome and timetable

    Animesh: sharing a Google Doc with the above
 information

    Maureen McElaney (IBM): 
I’ve added a loose agenda for today’s call to the wiki. Since I’m on mobile can someone take on the responsibility of adding any further notes there?

    Jim Spohrer (IBM): I am taking notes and can clean them up and upload them, link from Wiki to notes

    
Animesh: Introducing Romeo Kienzler (IBM Switzerland) who added LF AI meeting in Paris hosted by Orange
    Animesh: Romeo has created top data sciences and AI Coursera courses

    Alka Roy (AT&T Innovation Center) - 7:15 AM PT - Had trouble with new link - just joining.
    
Romeo Kienzler (IBM Switzerland): Working on pipelines for technical working group

    Animesh: Can we use Slack to keep an ongoing work cadence
?
    
Nat: Not slack because of licensing complications. ICRC is being used instead. Communication channel gap.
    
Nat: Acumos has SCRUM meetings almost everyday

    Nat: Acumos meetings - we can re-use time slots and leverage for Trusted AI WG.
    Nat: Tencent Angel team interest in on boarding a model

    
Nat: two working groups - Principles and Technical
    
Nat: Synergies between Acumos + Angel + AIF360
    
Nat: Principles works on policies and strategies


    Alka Roy (AT&T Innovation Center) - 7:30 AM PT: Good morning!

    Alka: How to get to a set of criteria and guidelines?
    Animesh: job of WG Principles
    
Alka: Principles team creates, and technical team provides feedback/adopts
    
Animesh: yes, criteria evolves with feedback from terchnical


    Nat Subramanian (Tech Mahindra):: Romeo is IBM in Switzerland?

    Nat: China has most difficulty joining with times we have proposed so far (early California)

    Animesh: Romeo’s time might be better for China and India meetings

    
Animesh: Let’s identify right subset of people, and let’ them find a good weekly time for them; Since Nat knows the folks - he can best find a way to loop in Romeo. I am OK with calls after 10pm PT

    Nat: I will get a time for Technical WG

    
Animesh: I will add Romeo email to WIki - his email is 
romeo.kienzler@ch.ibm.com

    Francois Jezequel (Orange) - 7:32 AM PT: Suoad is willing to continue, just not available today

    Animesh: Souad
    Francois: “Sue-wad” rhymes with “Squad”
    Alka: IEEE has good work we can borrow from

    
Alka: how to apply principles, practical technical and principle WG interactions

    Alka: going beyond theory into practice will be helpful
    
Animesh: Required people and optional people

    Animesh: Lineage -> Certification -> Badging

    
Animesh: Can help connect with IBM Research and ATT and rest of Principles committee

    
Animesh: Factsheets project is relevant - Factsheets: Increasing Trust in AI Services Through Supplier Documents of Conformity - Certifications - https://arxiv.org/pdf/1808.07261.pdfhttps://www.ibm.com/blogs/research/2018/08/factsheets-ai/
    
Alka: Please add me to the Principles (optional), and I will find the right ATT required

    Animesh: Alka is Bay Area for a visit
 sometime to discuss
    Alka: Rueben (AT&T)
    
Alka: I will make sure AT&T is engaged

    Nat: I have met with Rueben in NJ

    
Jeff Cao (Tencent) - 7:44 AM PT: I have ideas for person on technical person - Fitz Wang (Tencent)
    
Nat: I will reach out to Fitz, and please share name of others


    Mikael Anneroth (Ericsson) - 7:45 AM PT: please add Ericsson to Principles work group - would like to share and discuss - I am in Stockholm Sweden

    Jeff Cao (Tencent): Han Xiao is on the list

    
Animesh: Susan Malaika (IBM) is also interested)

    Animesh: If Orange will drive Principles

    Jeff Cao (Tencent): Tencent can join the principles group


    
Francois Jezequel (Orange): Yes, Souad is interested in leading Principles WG
    
Alka: identify gaps since a lot of work has already bee done

    
Francois: Minimum intersection that can be operationalized (how to make it practical in operation)

    Alka: Please let’s make the goal in writing, and when to have meeting to share each perspective, consolidate, and agree

    
Animesh: Francois or Souad can send a kick-off email, and we can get each group add their members (like IBM include Susan, Maureen, Jim, etc.

    Animesh: Shared several Google doc links and will upload them to Wiki

    
Alka: Strategy meeting - f2f being planned - in October - by that time converged on principles - progress to report

    Ofer: End of October, OSS Oct 31, full day governing board meeting
    
Nat: details readout in Oct 31
    
Alka: good deadline

    
Alka: Trusted and Responsible AI added to mission
 for LF AI - any objections
    All: No objections noted

  3. Working Group Names and Leads:

    Principles, lead: Souad Ouali (Orange France) with members from Orange, AT&T, Tech Mahindra, Tencent, IBM, Ericsson, Amdocs.


    Technical, lead: Romeo Kienzler (IBM Switzerland) with members from IBM, AT&T, Tech Mahindra, Tencent, Ericsson, Amdocs, Orange.


    Working groups will have a weekly meeting to make progress.  First read out to LF AI governing board will be Oct 31 in Lyon France.

    The Principles team will study the existing material from companies, governments, and professional associations (IEEE), and some up with set that can be shared with the technical team for feedback as a first step.  We need to identify and compile the existing materials.

    The Technical team is working on Acuomos+Angel+AIF360 integration demonstration.

  4. Thanks Souad for your note below to committee members - just re-reading it...

    (1) Great to keep a global balance on the committee

    (2) Look forward to definitions you are drafting

    (3) Collecting material - see some links below.

    (4) Cross analysis - Harvard study good starting point.

    (5) Pragmatic approach - what real-world enterprise use cases?

    (6) Open source tools - how tools and Trusted AI Workflows relate to above use cases?

    (7) Watch on LFAI landscape projects - which projects have been checked against principles?

    (8) How to ensure trust in open source package - how to organize audit to ensure files not corrupted? See Factsheets link below for a start.


    Some useful link:

    Globally Diverse - LFAI Trusted AI Committee: https://wiki.lfai.foundation/display/DL/Trusted+AI+Committee

    University: Harvard AI Principles meta-analysis: 32 sets of principles side by side, with 8 themes: https://ai-hr.cyber.harvard.edu/primp-viz.htm

    Company: AT&T (Tom Moore, Chief Privacy Officer): https://about.att.com/innovationblog/2019/05/our_guiding_principles.html

    Company: IBM Everyday Ethics for AI: https://www.ibm.com/watson/assets/duo/pdf/everydayethics.pdf

    Company: IBM NIST AI Standards response: https://www.nist.gov/sites/default/files/documents/2019/06/06/nist-ai-rfi-ibm-001.pdf

    Company: IBM Factsheets: Increasing Trust in AI Services Through Supplier Documents of Conformity - Certifications - https://arxiv.org/pdf/1808.07261.pdf

    Company: Tencent sent some files - but I do not have a URL to the documents.

    Trusted AI group calendar: https://wiki.lfai.foundation/pages/viewpage.action?pageId=12091895 (Please subscribe)
    Mailing list: https://lists.lfai.foundation/g/trustedai-committee (Please subscribe)
    GitHub: https://github.com/lfai/trusted-ai/


    Q: Where is the best place for us to upload documents to share with the Trusted AI Committee/Principles WG?

  5. Some notes from Oct 17, 2019 committee call:


    Summary
    - Animesh walked through the draft slides
    (to be presented in Lyon to LFAI governing board about TAIC)
    - Discussion of changes to make
    - Discussion of members, processes, and schedules

    Detail
    - Jim will put slides in Google Doc and share with all participants
    (https://drive.google.com/drive/folders/1RSHBzTj7SRpbioR31JP9ASEjvuV9kqnw)

    - Susan is exploring a slack channel for communications
    - Trust and Responsibility, Color, Icons to add Amdocs, Alejandro's Institute
    


    From chat:
    + @Ofer to PWG

    Alka: helped draft these - and acknowledges need to get everyone’s perspective - asked about feedback loop with Usecases Group

    
Francois: These slides will be a good resource - thank-you; now we have to organize the way we contribute to each part of the document - Suoad will lead getting consensus and finalizing this draft

    
Francois: Still need to define connection between principles and tooling - and make it concrete
Animesh: Members of UWG?
Animesh: Need members who will create use cases - software and actual industry use cases
Animesh: What are the telco use cases?

    
Alka: personal focus on PWG definition. Will work to gather AT&T use cases and try to find the right active person

    
Nat: interest yes. However, current focus on the release - in 3-4 weeks from now release will be complete for Acumos. Technical people tied up on release.
Nat: Reuben can give guidance, and Nat working on next level of technical experts

    
Han: Jeff and I working to get more awareness and attract others in Tencent; open source office and develops team to get involved. Find real world use cases

    Francois: Discussion of when PWG will meet - off-line and maybe two meetings. (Alka, Francois, Jeff, Susan)
    Alka: I have material to share with everyone when we can schedule that
    Animesh: Alejandro has great material on GitHub.