Overview

During the march 11th GB F2F meeting, the board recommended having a reference ML workflow / stack, and showcasing where current LF AI hosted projects fall. The idea is to have a reference architecture and to allow us to identify gaps in our project portfolio so we can focus on project recruitment in areas that we need most, leading us to provide a reference implementation using LF AI hosted projects.

Mailing List

Please subscribe to the ML Workflow mailing list: https://lists.lfai.foundation/g/mlworkflow-committee

Meetings

Every 4 weeks on Thursday, 10am EST right after the TAC meeting.

To get invites, please join the mailing list

Zoom: https://zoom.us/j/561494663

Past Meetings - 

Meeting Content (minutes / recording / slides / other):


DateMinutes

 

Agenda for the meeting:

1. Update about the integration between Acumos and other LFAI projects - Angel and ONNX
2. Review the latest version of the ML Workflow slides and map
3. Discuss our focus going forward
Agenda for the meeting:

Minutes:

  • We have now a dedicated mailing list, please make sure you register so you can get all the emails and invites
  • Ofer Hermoni shortly explained to goals of the committee
  • We mapped the new project Sparklyr on the ML stack, and updated the deck
  • Natarajan Subramanian and Fitz Wang updated regarding the integration between Angel and Acumos - the technical teams are working together on the integration
  • Natarajan Subramanian updated re the status of the integration between Acumos and ONNX. Jim Spohrer said he will dedicate more resources to help with that
  • Logistics:
    • Everyone should register to the mailing list in order to get the invites
    • Meetings will be scheduled for once in 4 weeks on Thursdays 10am EST (immediately after the TAC call)
  • We discussed the goals of this committee:
    • The integration between the different projects will be done by the technical teams of the projects. This committee will support that and identify potential collaboration opportunities and identify resource gaps
  • Jim Spohrer suggested to work with the Red Hat team on the Open Data Hub initiative. He will invite the leader of this initiative to talk in the next meeting or so

 

  • Goals defined for the meeting:
    • Reevaluate the ML Workflow stack according to feedback we got
    • Work on integration between different LFAI projects
  • Ofer Hermoni shared that there is a lot of interest from the community (LFAI members and the broader community), and many joined this effort, we have 18 people representing 8 companies!
  • evaluate the ML Workflow stack (see here updated deck):
    • Patrick Fu (from Gemini Open Cloud) shared their view on the ML Pipeline (see slide 2 on the deck), main difference was the data consistency layer - we added it to our stack
    • Ofer Hermoni mentioned that Fitz Wang showed in OSS that Angel supports Feature Engineering as well - we added that to the stack
    • Next step - update existing projects on the new stack
  • Integration between different projects:
    • Natarajan Subramanian shared with the team the work we are doing regarding the integration between Acumos and Angel and between Acumos and Horovod
    • Next step - set a "1:1" session dedicated to the integration between Angel and Acumos (Thursday September 19th)
  • More next steps

 

  • We farther refined the presentation and discussed next steps for this group
  • IBM presented some of the internal work they are doing around AI open source

 

  • We discussed the different open source projects and mapped them to our stack. See the results here
  • Next week Animesh will present a framework IBM is building and we will discuss how it can be beneficial to create collaboration between different LFAI projects
  • We will also prepare content for the next TAC meeting in which we will review our progress

 

  • We defined our goal, and started to review the stack. This is the result


Participants (archive as we moved to use the mailing list)

Committee Lead: Ofer Hermoni <oferher@gmail.com>

Name

Affiliation

Email

LF ID

Ofer HermoniAmdocsoferher@gmail.com  
Ibrahim HaddadLinux Foundationibrahim@linuxfoundation.org
Jim SpohrerIBMspohrer@us.ibm.com
Nat SubramanianTech Mahindranatarajan.subramanian@techmahindra.com
Ian LumbSylabs.ioilumb@sylabs.io
Jamil ChawkiOrangejamil.chawki@orange.com
Vishnu Ram OVMember of ITU FG ML5Gvishnu.n@ieee.org
Animesh SinghIBMsinghan@us.ibm.com
Vijay R BommireddipalliIBMvijayrb@us.ibm.com
Ganesh HarinathVerizonganesh.harinath@verizonmedia.com
Kiril NejkovIFCknejkov@ifc.org
Gordon I. MyersIFCgmyers@ifc.org
Guangji XueIFCgxue@ifc.org
Hussain A. AlkazemiWorld Bank Group (WBG)halkazemi@worldbankgroup.org
Patrick FuGemini Open Cloudpatfu2005@geminiopencloud.com
Edison Peng Gemini Open Cloudedison@geminiopencloud.com
Fitz WangTencentfitzwang@tencent.com
Clive CoxSeldoncc@seldon.io

Sophia Arakelyan

Simioticssophia@simiotics.com

Neeraj Kashyap

Simioticsneeraj@simiotics.com