There's quite a gap in between research and reality, said panelists at an NVIDIA GTC 2022 panel. The Leaders in AI panel session: AI journey from concept to scale discussed bridging research with the real world.
One way that research differs from practice include risk appetite, said Jure Leskovec, Associate Professor & Senior Technical Advisor, Stanford University & Pinterest. In an academic setting, students can experiment more freely than corporate researchers, he noted. "How do you manage the risk of cutting-edge technology and manage that in the enterprise? It requires a lot of thinking.. a lot of planning, ...a lot of science, to take it and deploy it in terms of scale," he shared.
In the real world, how "deployment must occur at a larger scale than what we would typically consider in a research paper", Nicolas Chapados, VP Research, ServiceNow said. He talked about the challenge of training an AI model with enough language data to support 34 languages for a project as an example in his experience.
Another example of scale, Chapados added, involves speed. This can be controlled for a small number of users, but things become more complex when there may be 10,000 users, all of whom must be supported with "acceptable latency".
Scale is a challenge, agreed Ya Xu, VP of Engineering, Head of Data, LinkedIn, listing latency as well as cost and hardware limitations as possible obstacles. At LinkedIn, recommendations need to be returned in hundreds of milliseconds, she shared. Her solution is to have multiple ranking layers, with an initial ranking layer featuring less sophisticated modelling. Another way to tackle latency is to distill (compress) the existing model into a smaller one, but performance might be affected, she said.
AI observability - linking what is happening inside a system based on data produced by that system - has also got to be in better shape, Xu continued. She explained that when something goes wrong, good-quality data is important, as is implementing the right continuous integration/continuous delivery (CI/CD) system for incremental updates.
Xu further highlighted the significance of goal definition. While AI models can be optimised towards a particular goal, what that goal is and how it translates into the real world is important, she said. This point was taken up by Chapados, who shared that at ServiceNow there is a research transfer team that creates proofs of concept and proofs of value based on ServiceNow concepts and data. These projects yield tangible metrics that help the company make an informed judgment about how a research advancement will shape the product and platform.
On the non-technical side, organising the team, ensuring it has the right culture and collaborative partnerships is crucial, she concluded.
Communication can be a problem as well, Chapados said, highlighting that there are many people in the supply chain who could be using the same terms to mean very different things. He called establishing standards for communication a non-trivial issue. Bryan Catanzaro, VP, Applied Deep Learning Research, NVIDIA, recounted how 'performance' can mean 'faster' for some segments of NVIDIA and 'more accurate' for those who work in AI. He prefers his team to avoid certain terms so that there is less misunderstanding.
Mismatches in expectations on how things are executed and measured can also occur, Chapados said. On this point, it is a misconception that third party AI models can be deployed 'as is' in a particular environment, Leskovec added. "You cannot just download someone else''s model and deploy it," he said, calling the thought a "losing proposition". "You have to understand it, modify it and then deploy it (to see a) 20%-30% uplift in benefits.
Catanzaro, commented that there can be amazing prototypes that "don't actually work when deployed". He suggested that ideas have to flow bidirectionally, with collaboration fostered between researchers and companies together with the product teams which are building products, as these are the people who are actually deploying the technology and understand the real-world constraints about deployment.
"Research teams need to learn from the product teams about what actually works in the real world, and that requires some pretty tight collaborations," he said. His thoughts were echoed by Xu, who called for "stronger and tighter" collaboration between "folks who do research and folks who do production".
Chapados agreed, saying that bringing researchers together with product managers can show the researchers the "difficult use cases", what problems customers are truly facing. "Then creative juices start flowing," he said. Interactions also flow the other way, with researchers talking to the broader organisation about advances in technology that could change the way a problem is viewed.
Leskovec pointed out that the research process is typically undertaken with the expectation that there is a "customer on the other side", as opposed to building something that works for no one.
Explore:
To view a replay of the session, search for S42554 at the GTC microsite (registration required)
No comments:
Post a Comment