During the march 11th GB F2F meeting, the board The LF AI Governing Board recommended having a reference ML workflow / stack, and showcasing where current LF AI hosted projects fall. The idea is to have a reference architecture and to allow us to identify gaps in our project portfolio so we can focus on project recruitment in areas that we need most, leading us to provide a reference implementation using LF AI hosted projects.
Howard Huang of Huawei presented a proposal for interoperability between the different LFAI projects and also external projects
We had a discussion if there is a need for a real interoperability effort or maybe a standard will be enough? We agreed that there is a need for a real interoperability effort
Next steps:
In the next ML Workflow meeting Howard will present a plan for the effort
In one of the next TAC meetings Howard will share the proposal with the broader team
We reviewed the current version of the ML Workflow, and tried to identify the correct blocks for the two new projects (Milvus and Marquez)
Jun of Zilliz will think about that for Milvus
Ofer Hermoni will approach the WeWork team to identify the right location for Marquez
Project integration:
Natarajan Subramanian covered the status of the collaboration / integration between Acumos and Angel, and Acumos and Adlik. Due to the Corona epidemic everything is slowed down currently in China
Natarajan Subramanian also mentioned the work Acumos is doing with AI360. Currently in discussions phase
Jessica Kim shared the status of the integration infrastructure Huawei has in Hong Kong, and the plan to add more locations based on Tencent public cloud - both in China and Canada
Jessica Kim suggested to leverage the integrations and the ML Workflow activity as part of the outreach activity
In this meeting we had a team from RedHat to present ODH (Open Data Hub)
Apparently there are many overlaps between the two initiatives
Next steps
Invite the ODH team to present to the TAC
A representative of the TAC will join one of the ODH meetings to present LFAI and ML Workflow
Agenda for the meeting:
1. Update about the integration between Acumos and other LFAI projects - Angel and ONNX 2. Review the latest version of the ML Workflow slides and map 3. Discuss our focus going forward Agenda for the meeting:
Minutes:
We have now a dedicated mailing list, please make sure you register so you can get all the emails and invites
Ofer Hermoni shortly explained to goals of the committee
We mapped the new project Sparklyr on the ML stack, and updated the deck
Natarajan Subramanian and Fitz Wang updated regarding the integration between Angel and Acumos - the technical teams are working together on the integration
Natarajan Subramanian updated re the status of the integration between Acumos and ONNX. Jim Spohrer said he will dedicate more resources to help with that
Logistics:
Everyone should register to the mailing list in order to get the invites
Meetings will be scheduled for once in 4 weeks on Thursdays 10am EST (immediately after the TAC call)
We discussed the goals of this committee:
The integration between the different projects will be done by the technical teams of the projects. This committee will support that and identify potential collaboration opportunities and identify resource gaps
Jim Spohrer suggested to work with the Red Hat team on the Open Data Hub initiative. He will invite Sherard Griffin to talk in the next meeting
Jim Spohrer encourages collaboration and integration to IBM AI Fairness framework, it will help to convince IBM to contribute this framework of projects to LFAI
Goals defined for the meeting:
Reevaluate the ML Workflow stack according to feedback we got
Work on integration between different LFAI projects
Ofer Hermoni shared that there is a lot of interest from the community (LFAI members and the broader community), and many joined this effort, we have 18 people representing 8 companies!
Patrick Fu (from Gemini Open Cloud) shared their view on the ML Pipeline (see slide 2 on the deck), main difference was the data consistency layer - we added it to our stack
Ofer Hermoni mentioned that Fitz Wang showed in OSS that Angel supports Feature Engineering as well - we added that to the stack
Next step - update existing projects on the new stack
Integration between different projects:
Natarajan Subramanian shared with the team the work we are doing regarding the integration between Acumos and Angel and between Acumos and Horovod
Next step - set a "1:1" session dedicated to the integration between Angel and Acumos (Thursday September 19th)
We discussed the different open source projects and mapped them to our stack. See the results here
Next week Animesh will present a framework IBM is building and we will discuss how it can be beneficial to create collaboration between different LFAI projects
We will also prepare content for the next TAC meeting in which we will review our progress