Overview

During the march 11th GB F2F meeting, the board recommended having a reference ML workflow / stack, and showcasing where current LF AI hosted projects fall. The idea is to have a reference architecture and to allow us to identify gaps in our project portfolio so we can focus on project recruitment in areas that we need most, leading us to provide a reference implementation using LF AI hosted projects.

Mailing List

Please subscribe to the ML Workflow mailing list: https://lists.lfai.foundation/g/mlworkflow-committee

Meetings

Every 4 weeks on Thursday, 10am EST right after the TAC meeting.

To get invites, please join the mailing list

Zoom:https://zoom.us/j/9918615568


Agenda for 02/13/2020 meeting:

  1. Review our blueprint and add all new projects
  2. Update on integration between projects
  3. Follow up discussion on Huawei initiative to run a workgroup for integration


Past Meetings - 

Meeting Content (minutes / recording / slides / other):


DateMinutes

 

Call was recorded, this is the record. And this is the chat.

In the meeting we covered the following:

  • We reviewed the current version of the ML Workflow, and tried to identify the correct blocks for the two new projects (Milvus and Marquez)
    • Jun of Zilliz will think about that for Milvus
    • Ofer Hermoni will approach the WeWork team to identify the right location for Marquez
  • Project integration:
    • Natarajan Subramanian covered the status of the collaboration / integration between Acumos and Angel, and Acumos and Adlik. Due to the Corona epidemic everything is slowed down currently in China
    • Natarajan Subramanian also mentioned the work Acumos is doing with AI360. Currently in discussions phase
  • Jessica Kim shared the status of the integration infrastructure Huawei has in Hong Kong, and the plan to add more locations based on Tencent public cloud - both in China and Canada
  • Jessica Kim suggested to leverage the integrations and the ML Workflow activity as part of the outreach activity

 

In this meeting we had a team from RedHat to present ODH (Open Data Hub)

  • In this meeting we started recording our calls. The Zoom recording can be find here. The chat here
  • Deck presented is available here
  • Ofer Hermoni gave a background about the ML Workflow working group goals and activity
  • Redhat team provided a presentation about ODH
  • Apparently there are many overlaps between the two initiatives
  • Next steps
    • Invite the ODH team to present to the TAC
    • A representative of the TAC will join one of the ODH meetings to present LFAI and ML Workflow

 

Agenda for the meeting:

1. Update about the integration between Acumos and other LFAI projects - Angel and ONNX
2. Review the latest version of the ML Workflow slides and map
3. Discuss our focus going forward
Agenda for the meeting:

Minutes:

  • We have now a dedicated mailing list, please make sure you register so you can get all the emails and invites
  • Ofer Hermoni shortly explained to goals of the committee
  • We mapped the new project Sparklyr on the ML stack, and updated the deck
  • Natarajan Subramanian and Fitz Wang updated regarding the integration between Angel and Acumos - the technical teams are working together on the integration
  • Natarajan Subramanian updated re the status of the integration between Acumos and ONNX. Jim Spohrer said he will dedicate more resources to help with that
  • Logistics:
    • Everyone should register to the mailing list in order to get the invites
    • Meetings will be scheduled for once in 4 weeks on Thursdays 10am EST (immediately after the TAC call)
  • We discussed the goals of this committee:
    • The integration between the different projects will be done by the technical teams of the projects. This committee will support that and identify potential collaboration opportunities and identify resource gaps
  • Jim Spohrer suggested to work with the Red Hat team on the Open Data Hub initiative. He will invite Sherard Griffin to talk in the next meeting
  • Jim Spohrer encourages collaboration and integration to IBM AI Fairness framework, it will help to convince IBM to contribute this framework of projects to LFAI

 

  • Goals defined for the meeting:
    • Reevaluate the ML Workflow stack according to feedback we got
    • Work on integration between different LFAI projects
  • Ofer Hermoni shared that there is a lot of interest from the community (LFAI members and the broader community), and many joined this effort, we have 18 people representing 8 companies!
  • evaluate the ML Workflow stack (see here updated deck):
    • Patrick Fu (from Gemini Open Cloud) shared their view on the ML Pipeline (see slide 2 on the deck), main difference was the data consistency layer - we added it to our stack
    • Ofer Hermoni mentioned that Fitz Wang showed in OSS that Angel supports Feature Engineering as well - we added that to the stack
    • Next step - update existing projects on the new stack
  • Integration between different projects:
    • Natarajan Subramanian shared with the team the work we are doing regarding the integration between Acumos and Angel and between Acumos and Horovod
    • Next step - set a "1:1" session dedicated to the integration between Angel and Acumos (Thursday September 19th)
  • More next steps

 

  • We farther refined the presentation and discussed next steps for this group
  • IBM presented some of the internal work they are doing around AI open source

 

  • We discussed the different open source projects and mapped them to our stack. See the results here
  • Next week Animesh will present a framework IBM is building and we will discuss how it can be beneficial to create collaboration between different LFAI projects
  • We will also prepare content for the next TAC meeting in which we will review our progress

 

  • We defined our goal, and started to review the stack. This is the result


Participants (archive as we moved to use the mailing list)

Committee Lead: Ofer Hermoni <oferher@gmail.com>

Name

Affiliation

Email

LF ID

Ofer HermoniAmdocsoferher@gmail.com  
Ibrahim HaddadLinux Foundationibrahim@linuxfoundation.org
Jim SpohrerIBMspohrer@us.ibm.com
Nat SubramanianTech Mahindranatarajan.subramanian@techmahindra.com
Ian LumbSylabs.ioilumb@sylabs.io
Jamil ChawkiOrangejamil.chawki@orange.com
Vishnu Ram OVMember of ITU FG ML5Gvishnu.n@ieee.org
Animesh SinghIBMsinghan@us.ibm.com
Vijay R BommireddipalliIBMvijayrb@us.ibm.com
Ganesh HarinathVerizonganesh.harinath@verizonmedia.com
Kiril NejkovIFCknejkov@ifc.org
Gordon I. MyersIFCgmyers@ifc.org
Guangji XueIFCgxue@ifc.org
Hussain A. AlkazemiWorld Bank Group (WBG)halkazemi@worldbankgroup.org
Patrick FuGemini Open Cloudpatfu2005@geminiopencloud.com
Edison Peng Gemini Open Cloudedison@geminiopencloud.com
Fitz WangTencentfitzwang@tencent.com
Clive CoxSeldoncc@seldon.io

Sophia Arakelyan

Simioticssophia@simiotics.com

Neeraj Kashyap

Simioticsneeraj@simiotics.com