Viam is a robotics platform that brings modern software development tools into hardware applications.
Connect with Simone on Linkedin.
This week’s shoutout goes to Lifejacket winner Sergey Kalinichenko for their answer to How does this K&R code for reading an int work?.
TRANSCRIPT
Ryan Donovan: Working with voice? Try Assembly AI instead of wrestling with DIY models. You get insanely accurate transcription and diarization across 99 languages – all in one API that plugs right into your stack. Start building with $50 in free credits at assemblyai.com/stackoverflow.
[Intro Music]
Ryan Donovan: Hello everyone, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I am your humble host, Ryan Donovan, and today, we're talking about robots and the disconnect between the hardware and software in AI. My guest today is Simone Kalmakis, VP of engineering at Viam. So, welcome to the show, Simone.
Simone Kalmakis: Thanks so much, Ryan, for having me.
Ryan Donovan: My pleasure. Now, I know we've had Elliot Horowitz, founder of the company on before, but we haven't had you on before. So, top of the show, we like to get to know our guests. Tell us a little bit about how you got into software and technology.
Simone Kalmakis: I come from a family of Luddites, so we're talking lawyers, we're talking doctors and scientists, you know, real scientists, not computer scientists like in my field. And from an early age, I was always really the de facto tech support for my family – swapping out motherboards, I got really excited about technology. And then, that carried into college, where my job in college [was] IT tech support for the other students, and I like to say that I've never in my whole career had a more stressed-out customer than a Yale student whose term paper is on an ill-fated hard drive. I mean, just never anything like it. So, that was a good start. I went out to Microsoft first as an intern, and then for full-time. I was there for six years, and I worked on the core relevance of the search engine for Bing. So, dating myself already, but back then, when you typed in a search query, we used to show 10 blue links, and that was that. Things have changed. But that was really my start to data machine learning, which was really exciting for me as a background. After Microsoft, I went on to found my own startup called Symbi, which was matching people together based on compatibility as roommates. Very similar to online dating, but different type of compatibility. We ended up selling that startup, and I worked in a number of verticals, all focused on using data for how to move our society forward. So, I worked in ad tech; I worked in health tech; I worked for six years for Flatiron Health in New York City, focusing on developing drugs and treatments for cancer patients using data; I worked in climate tech at Arcadia Startup in DC; and now, I've been at Viam for a little over two years.
Ryan Donovan: Viam is [an] interesting premise. It's a build as the abstraction layer for AI robots. Obviously, with AI and AI agents, everybody's talking about higher abstraction layers and automation. Can you tell me about how that layer works at Viam?
Simone Kalmakis: Absolutely. So yeah, as you said, we are a software platform for robotics, and our goal is to bring the software development toolkit that we all are used to using to develop and iterate into the world of robotics. So, helping someone quickly and easily go from prototype to production, and then to scale further.
Ryan Donovan: You know, it almost sounds like either it's service meshes for robots, or like CICD. What's the sort of software engineering paradigm that you're bringing to robots?
Simone Kalmakis: Yeah, really an abstraction layer, an operating system– I mean, the biggest value of Viam is our breath. So, we have a modular system where a module might consist of, basically, a driver for a hardware component, all the way through to some code, a service that provides some sort of business logic, up to a machine learning model. Anything in between. So, we allow a user to be able to access any of these little bits and compose them together as building blocks, and that's really the key to what makes developing on Viam quick, iterable, you know, really easily get you from that first idea to a working solution in a fraction of the time that the current system solutions do.
Ryan Donovan: It almost sounds like you're going for a no-code. How composable, configurable, streamlined, are these little components to plug and play?
Simone Kalmakis: Yeah, it really depends. So, first of all, I think that you can build in a no-code way on top of Viam. We also are trying to be quite flexible to be able to be serviceable to a wide range of use cases, from individual engineer, to a small startup, all the way up to production level, like industrial robots. So, such a big range here is really our goal. And so, to your question, you can write your own module; you can upload it to a modular registry that we have; you can quickly use someone else's model or service that they've uploaded, as well; you can use something that Viam has built. The idea is flexibility, so you could easily grab the components you need for your system, or you could even write custom logic yourself and add that as another module. And that's the power of this module or building block system is a wide range of uses are possible.
Ryan Donovan: I imagine when somebody's designing their robot system, it's a lot of very custom hardware. Having this abstraction layer on top, how does that interface with what could be any sort of hardware on the– what is it, a backend, front end, with a robot? What do you call it?
Simone Kalmakis: In the physical system, exactly. Yeah. So, we have a binary called Viam Server that you run on the main board of the machine itself, and then, as you said, you'll have hardware components that integrate with that. And as you said, you know, one of the key components of building a robotic solution is you have these very varied solutions. They all have different components. They all are trying to do different things. And so this composability is so key to being able to work with that well, because once you have the software platform that treats them as composable—which is again, such an innovation of Viam's—now you can see the inputs and outputs to each resource reflected in software, and this is so helpful. Once you do build up a complex system, like you're saying, you usually– you know, a lot of robotic solutions, they start with an imaging or perception layer. You can actually see the output of what your camera is seeing via the Viam interface. Then, you need to see– you know, now we've taken, let's say, that depth image, we've turned it into what we can gather from those point clouds, we're segmenting. This is just the start of, let's say, a big robotics, longer, more complex system. You are able to interact with each of these pieces, easily debug, write unit tests against them – everything that you would rely on in the modern software toolkit, except what's novel is it was not really preexisting in the robotics world. So, you can iterate just as fast as you can in the software world.
Ryan Donovan: Do you have the benefit of knowing that there's gonna be either off-the-shelf pieces in robotics, hardware, or any kind of standards for interfacing with perception or movement?
Simone Kalmakis: We certainly try to provide support for the most common components, and then a key tenet of the system is that if you do come across a piece of hardware that is unsupported, you easily have the ability to add your own driver, add your own support for it, and then
Simone Kalmakis: Importantly, upload that to this registry, where other developers can take advantage of that. So, you're really creating this developer ecosystem with a flywheel that the more people who use it, the better the platform becomes.
Ryan Donovan: So, if there aren't standards or known entities, somebody is able to create that and upload it.
Simone Kalmakis: That's correct. Yeah, and one other aspect of the platform that I personally find very exciting as a software engineer is this notion of, essentially, inheritance, where we define a proto and API for, say, a camera, and any model of camera that you would buy has to, in the Viam module, conform to these methods. They conform to these; you can not implement something and specify that, but then, what's really powerful about that is when you're writing software on top of these components, you're just interacting with the methods via 'this API has specified.' And you don't have to be particular to the actual hardware model itself. And so, where this is so helpful is then you can easily swap out components, you can test different things. Oh– we did this with a robotic solution we're building at Viam recently, we swapped out a depth camera called a 'RealSense' for a depth camera called an 'Orbbec.' It performed better, and it was a two-line code change essentially, because you're just specifying, 'hey, by the way, this config is actually this hardware.' And so, making it that easy on the software side is so paramount to this iteration speed improvement.
Ryan Donovan: So, for something like that, the two-line, I'm sure, is in the abstraction layer, but to get that, you're having to look at hardware interrupts, or are you having to read raw memory dumps on the robot system, anything like that?
Simone Kalmakis: So, as a user, you should not be having to do any of that, and this is really another advantage of the Viam platform is a lot of that rote code, error handling, alerting– you know, there's so many pieces that, currently, anyone building a robotic solution, like, say you're a robotic startup, you're two people, you're trying to get funding, you have to spend so much time creating this boiler plate code to just handle how to integrate with robotics components. You should be spending your time developing the idea that you created the startup for in the first place, and not this rote boilerplate code that everybody would have to develop in parallel. So, really abstracting that out, taking care of it for you is a big reason to come to Viam.
Ryan Donovan: Yeah, I mean, I was writing something today about platform engineering, and this sort of sounds like platform engineering for robots. Like you said, Viam handles the soup to nuts. Is it everything besides the sort of business logic of the robot, like the security, the reliability, all that stuff?
Simone Kalmakis: Exactly right. So, the platform takes care of that for you. Security off, you know, retries, reconfigures. You have to tell the system how you want your modules to behave, but all of that should be under the hood. Exactly. And then your job as the developer now becomes the hard work of iterating, getting things to actually work in the real world. You know, I mentioned earlier, we talked about my bio – this is really my first time working a lot with robotics, with hardware. I had done a few hobbyist projects previously, but really just more pure software, and it's been really eye-opening, just how different the engineering process is to get a solution actually working. And that's where we really want engineers to be spending their time: try this out, iterate, debug easily, we'll make it as easy for you as we can, and then you really need to spend those hours actually getting your solution to work, because so often it's not a subtle logic bug; it's more often like, 'oh, the lighting was off, so now your camera's images are not representative of how you trained your model.' It's more these environmental factors, and that's where the hours should be focused, not on replicating this functionality that would be agnostic to solution.
Ryan Donovan: I'm wondering, you know, how much of that software engineering practice you've tried to build into robotics, and how much you're able to, because we talk about iterative processes with robotics – there's a physical part that has to exist, so iterating is a little bit more costly, right?
Simone Kalmakis: Yeah, that's right. And that's been, as I said, eye-opening, as well. For me, coming down into the hardware world, the type of iterations are quite different. It's really around assessing the physical environment and really considering a whole new set of variables that could be at play here.
Ryan Donovan: So, what are the sort of missing software practices that you've tried to apply?
Simone Kalmakis: I mentioned already, you know, this API layer. I think that's a huge innovation because that swap ability – use the right robotic arm for the job, use something very small for certain tasks, use something very big and powerful for other tasks, or even maybe a given solution, might need two of them for different times and different situations. Then there's logging, alerting. We talk about this, like– imagine you're a 1998 Curl developer and someone hands you AWS – how much faster could you ramp up on that and go? Right now, we're kind of in that 1998 world in the world of robotics, and what you need is AWS, right? So, with Viam, you have sleep management. I talked about how important it is to be able to scale your solution, so now you can look across your fleet of machines. We have a notion called 'Fragments,' where you're basically specifying, 'this is the config that is common to my whole fleet of robots. Here, you can override that for a specific machine. Oh, this one over here happens to have this different camera, or whatever.' That's like a one-line change. And so, now when you're talking about versioning and upgrading your fleet with the newest changes you've made, you do that once you deploy that to your entire fleet, and you can monitor and see how it is going across your whole fleet. And again, kind of like the AWS analogy of we're all very used to this now in the web development world, you can now build a website in a good 10-20 minutes of work instead of days, weeks, months. In the robotics world, these concepts were sorely lacking.
Ryan Donovan: In the robotics world, though, you have a much more resource-constrained environment. Has it been difficult to adapt your own mindset of engineering to this very much more of a famine, right? You said we have a feast with the AWS, and the like. Has it been difficult to adapt your mindset to the resource constraints of these robotic devices?
Simone Kalmakis: That's what Viam really helps with is giving you that tooling that was really missing prior to it. And so, instead of just flying blind, you actually have insight into how these components are working. Yeah. But I did a project recently that I think is really illustrative of this. One thing we try to do at Viam is really encourage 20% projects, you know, the concept from Google, originally, of: 20% of your time should be working on your own project in service of making you better at your day job. One project I did this summer that was exciting to say the least, was my in-laws, who live in Gloucester, Massachusetts – they have a beautiful house on the water, and my father-in-law was not catching any lobsters in his lobster trap, especially relative to neighbors, which is infuriating. And who wouldn't wanna eat lobster? So, I said, 'I think Viam can help with this.' You know, we do a lot in the marine space, so this really had me inspired. So, we ended up putting a GoPro on his lobster trap, you know, sending it down in the water, having some waterproof cabling, et cetera, and capturing data off of what was happening underneath the ocean. And this is also a good example of, I've learned in robotics that the development cycle is so different. You know, this is true to some degree in software as well, but really in hardware, where your official theory can be quite off from what you end up needing to build to achieve that. So, I thought I was gonna be building this lobster notification system. I thought the problem was lobsters were coming in and out. My father-in-law didn't know. He checked the trap once a week, no lobsters there. Oh no. I was gonna have Viam text my father-in-law, you know, 'lobster detected.' No, it turns out as soon as we just simply integrated with a camera to get images off of what was happening, my initial assumptions were completely wrong. Actually, what was happening is as soon as the trap hit the ocean floor, it was inundated with about 20 crabs immediately, who were fighting each other for the bait. And I'll also add that I didn't realize that the camera was gonna be capturing audio, so I now have gigabytes of nightmare fuel footage of pincers – it's absolutely horrifying. I've terrified everybody. The point being that even just that initial step of adding images, essentially like adding eyes to what is actually happening, was incredibly illustrative, and specified a whole different set of actions and solutions than we had initially thought. Things like: we had actually configured– the trap was actually strung up in the wrong orientation of where the ropes were, we could immediately fix that; that was probably the wrong part of the ocean to have the trap in the first place, so we changed the location. The lobster notification system – I would've built that whole thing, and it would've just had true negatives all day long and done absolutely nothing for anybody. And so, really being able to interact and gather data from the physical environment to then build off of is such an important piece.
Ryan Donovan: Yeah. Lobster observability. Right?
Simone Kalmakis: Lobster observability, exactly. In my case, unfortunately, it ended up being more crab observability, but that was a key learning.
Ryan Donovan: Well, I'm sure there are FX engineers who would love to have that tape for some horror movie out there.
Simone Kalmakis: You don't need to add anything to it. It's already horrifying enough.
Ryan Donovan: That's interesting, that 20%. So, have you and your teams built other little test robots?
Simone Kalmakis: Yes, we've built a lot. We try to have a lot of robots around the Viam office. We had one that would grab coffee mugs, you know, just in case people weren't good about cleaning up their coffee mugs. I once saw a cake that had a Viam robot on it, where if the right person came, it would let them eat cake; otherwise, it wouldn't – it was doing facial detection. So, yeah, we just try to have a lot of robots around the Viam office. But you know, again, the power of the platform is really that it has so much flexibility, where you can build these more fun, hobbyist projects, or some with some function, like in this case, the lobster trap. But you can scale it all the way up to an industrial robot. I mean, some of our biggest use cases right now are at an industrial engineering level, and so I think that's really interesting, being able to have that flexibility of everything from a hobbyist project to something you really depend on, all the way up to being helpful at an industry level.
Ryan Donovan: One of the things I've talked about with IOT developers is updates and maintenance. Is there any sort of updating built in? Do you have robots with Bluetooth or wifi connections, and how difficult is that?
Simone Kalmakis: We are building support for over-the-air updates. You know, ideally this should be robust to a wide variety of conditions, so whether you're on wifi, whether you're on Bluetooth. We work a lot with marine use cases where your internet could be quite spotty, you might have no connectivity at all. We need to be quite robust to all of these solutions. We have a feature called Viam Agent, which allows for very easy continuous development and being able to push out new updates to a module or even a Viam server quite seamlessly. And again, the goal with the user not having to worry about any of this and abstracting the hard parts away from the user.
Ryan Donovan: So, I jokingly talked to other AI and robotics companies – my sort of go-to nightmare scenario is to Terminator. But I think for this one, there's an old cheesy Tom Selleck movie called Runaway, where tiny household robots go on killing sprees. So, with the over-the-air updates, how do you prevent malicious access to a robot?
Simone Kalmakis: Right, so this is where an authentication layer is so important. So, being able to make sure that the right API keys are in store, we take security extremely seriously, and so making sure that that authentication layer is built in very closely to the updates.
Ryan Donovan: You know, you talk about the industrial robots. What's the biggest, most complicated Viam implementation you've seen?
Simone Kalmakis: The most exciting one, which I'm very closely involved with, is with a customer called Viking Yachts, who is a boat-building company. So, they're building these large sport fishing yachts, and one problem they have is that a lot of their pieces are built out of fiberglass. Fiberglass goes into an injection molding process. When it comes out of the mold, it's very different from, you know, the CAD diagram that you had to go in due to the thermodynamic changes that happened during the injection molding process. So, a big step that you have to do then is use maybe 100 hours of manpower to just take a block sander, and do the very arduous labor of sanding this thing down so that it's smooth and homogenous, not just for aesthetics, but also, you know, this is going in the water.
Ryan Donovan: And fiberglass is not great to breathe in, right?
Simone Kalmakis: Well, I mean, that's a great point. So, not only are we talking about a hard job to do physically, but it's also dangerous for humans. It's dangerous to breathe in. It's dangerous to get in your pores. So, now imagine these people are all in full PPE, head to toe. They have to carry vacuums with them. And one other thing about fiberglass is it's flammable. So, now you have to worry about fire safety, insurance costs are through the roof, so many factors. And so, this is where there's really a big opportunity for robots to help. You know, robots can't be hurt by fiberglass in those ways, and so what we're building is a robotic sanding solution that is taking robotic arms and a block sander and finding the best way to sand the surface, and so then, you get the safety improvements. You get these health risks, [which] also leads to really high turnover for the companies, so from a business perspective, it helps them, as well. [It's] really one of these opportunities where a robotic solution just makes so much sense. You know, kind of like, I also like hearing about robots in hospitals 'cause they can't contract diseases the way humans do. So, anything where it's not only hard work, but also a dirty and dangerous job, I really think that's a great opportunity for robotic solutions. So, there's lots of interesting pieces to this puzzle. You know, first we image the piece itself and get back point clouds of where all of the points are in space. We merge these together, then we create a mesh, representing in a digital way, the surface that we've just imaged. Then we have to plan, 'okay, now in this image, let's segment it and determine where the strokes are going to go across the surface,' and then motion planning, which is: now that I know where I want the sander to contact the surface and how, now I need to get a robotic arm, which has six degrees of freedom into that path safely and efficiently. So, not a lot of exciting math to go into that. And then, at the end of it, repeat, you're continuing to image, and make sure that you know where the robot needs to go next. So, really complex end-to-end system with many points of failure, and that's where the composability of Viam helps the most.
Ryan Donovan: Can they add some kind of either generative AI or other modern newfangled AI to that composable system?
Simone Kalmakis: You certainly could. We haven't. Not for this solution in particular. Probably the most towards that is, you know, there's a lot of interesting statistical work with gradient descent in order to find the best paths. But yeah, more to your point, there's certainly other solutions. There's a startup that's built on Viam called Gambit, which is in the kitchen space, trying to build a, basically, chef robot. So, [to] help you as you're cooking, thermo sensors telling you, 'is this meat at this temperature? Is it ready?' Telling you when to flip the pancake, things like that. Very cool. And for them, they've integrated a whole LLM layer, so it's a completely auditory interaction layer where you're talking to the device, it's talking to you back. I think that's a really great use case for more of this generative AI.
Ryan Donovan: Yeah, the robotics stuff uses the more traditional machine learning, right? The vision is very big for the convolutional neural networks, and all that.
Simone Kalmakis: Yeah, absolutely. You see a lot of that in the vision models. Exactly. More traditional machine learning in the CV space. The versatility of the Viam platform is so interesting, where we haven't talked as much about some of the marine applications we've done, but really exciting how we can make a really resource-strained environment still feasible. So, imagine you have, you know, a small device with a very limited battery resource on Starlink, so pretty spotty Wifi at best. And this needs to be workable despite all of these constraints. So, I find that so interesting about working in the robotic space in particular, and what Viam tries to do, all the way down to a project that you could use in your day-to-day life, home automation, et cetera.
Ryan Donovan: It is that time of the show again where we shout out somebody who came on to Stack Overflow, dropped some knowledge, shared some curiosity, and earned themselves a badge. Today, we're shouting out the winner of a life jacket badge—somebody who found a question that was sinking with a score of negative two or less, and they dropped an answer that got five or more and brought the question up. So, congrats to Sergey Kalinichenko for answering 'K&R code for reading an int.' If you're curious about that, we'll have the answer in the show notes. I'm Ryan Donovan. I edit the blog, host the podcast here at Stack Overflow. If you have questions, concerns, topics to cover, et cetera, please email me at [email protected], and if you wanna reach out to me directly, you can find me on LinkedIn.
Simone Kalmakis: Thanks so much. I'm Simone Kalmakis, and you can find me on LinkedIn, as well.
Ryan Donovan: And how can they learn more about Viam?
Simone Kalmakis: Viam.com
Ryan Donovan: Well, we'll talk to you next time.