Dilma Da Silva: I see the number of participants is still going on. It's up, so let's wait until it gets a little stable. I guess
Dilma Da Silva: so. Hello, everyone
Dilma Da Silva: I'm Dilma da Silva. I am the division director for Ccf. Computing communication foundations in size, and here we size are delighted to have the opportunity to host a Professor Mike Carbine.
Dilma Da Silva: So Professor Carbine is an associate professor of electrical engineering and computer science at Mit, where he leads the programming systems group in his work. He aims at improving reliability, performance, energy, consumption, and resilience for computer systems, one hundred
Dilma Da Silva: using techniques from programming language. We are happy to say that Professor Kami has received an Nsf. Career. Award Islam Foundation Research Award Mit Frank Banks award for excellence in graduated devising.
Dilma Da Silva: His work has been recognized in so many forums, but one of those you know best paper awards for I Ilr and I cfp the
Dilma Da Silva: He got his undergrad in computer science from Stanford in two thousand and six, and then a Phd. In electoral and computer science from Mit.
Dilma Da Silva: And he then went to Microsoft Research, where he worked with deep learning systems from two thousand and fourteen to two thousand and eighteen.
Dilma Da Silva: So ah!
Dilma Da Silva: We are very pleased to have you here, Professor Carmine, Mike, and go ahead, and we are looking forward to learning a lot of programming,
Dilma Da Silva: uncertainty and the relationship between them.
Michael Carbin: Great. Thank you so much for that generous interaction.
Michael Carbin: So yeah, So today i'm going to talk about. I guess some work that I've been working on
Michael Carbin: since graduate school, I guess, coming on almost fifteen years now, and
Michael Carbin: the the flavor that has changed over the years as our computing systems have changed over the years. I mean just thinking about the past few months of what's going on with, you know. Chat your D. And G before, and just a new ways of thinking about software. It seems quite. I mean quite fantasical. It's a waste. Um, but for a long time it's It's sort of been what I've been expecting in some ways, because what I what I think and the way I think about the world where computation is going is that every single computation that we're dealing with and programming that or many of the very interesting, but emerging computations that we're thinking about are what I call a certain conversation,
Michael Carbin: and so to let me try to explain that over the course of this talk. I try to explain this, give some examples, and also sort of organize how we might think about the future software and important problems that we might need to be working on.
Michael Carbin: So what is an uncertain computation?
Michael Carbin: Well, the way I like to explain this these days is by looking at this cartoon diagram of the computing stack.
Michael Carbin: And the way I came to this explanation was, I was teaching a computer systems engineering class a couple years ago now, and I was. We were talking with students. I was talking with students about basically trust. You know, your trustee, What what parts of your system do you trust to be reliable.
Michael Carbin: Perhaps here are free or secure. You know what definitions and we're going through levels of the stack, and we started up with applications. The students said, Oh, applications! Libraries can't trust those I write code. I write out the they have bugs,
Michael Carbin: you know, compiler and runtime system. So a layer lower than that, you know It's not not necessarily so obvious to everyone that there there are bugs. Um that we need to deal with at those systems. But There's a long history of work, you know, to my community the Program Age community broadly,
Michael Carbin: and of course they put it wonderfully by and Nsf. On actually identifying bugs in these types of systems and reading them. How can we actually do secure computation
Michael Carbin: Now, below that operating systems? You know, I think all of us have had the unfortunate experience of dealing with operating system. Bug. These are windows, even back any of these things. But where things got interesting, and I was talking to the students asked about hardware systems, and he said, Oh, no, hardware. That's great,
Michael Carbin: you know. And this was before, you know. Perhaps the specter in meltdown, more security focus that perhaps we're seeing more broadly in the community around hardware security. But these again, were undergraduate students, and they said, Yes, you know if I have some instruction, it's one plus one. Then i'm going to get two as each, and I unfortunately
Michael Carbin: does not always the kiss.
Michael Carbin: And so This game is a shock, and it's a prize, and sometimes it comes to a shock. Is it so? People who don't really think about these issues? But, in fact, you right now you could go on the machine that you're sitting right now and go to Intel's website and find the errata, or whichever processor you know, sitting in your machine right now. Here's one from a few years ago.
Michael Carbin: And what is going to tell you? Actually, all the all, the all the bugs all the errors that may have it while executing programs or software on your system. Now, some of these may be rather benign. So this first one at the top cache on cache performance. Monitoring events need to be accurate.
Michael Carbin: You're doing low-level performance engineering. Maybe this this isn't something that is going to affect your day to day but this one below here at the bottom this this is a little bit more ambitious, so it says, processor may hang or cause unpredictable behavior and really unpredictable system behavior here. But what it means is that your system can return arbitrarily incorrect results at sometimes.
Michael Carbin: And this is not just, you know. In theory it's very, very rare circumstances, but this actually shows up in practice.
Michael Carbin: You're your consumer, and you're really hammering on your machine. You know consumer level really doing extensive computations. You can see these types of errors actually show up in corruptor stations. Now, if you go up another level and think about large massive organizations. They're running supercomputer and data set on their jobs. These types of errors actually are a primary first order concern in the design of these systems, because you can see that even within the course of a single day, you're very likely to see some sort of corruption inside your company
station,
Michael Carbin: and this is because we look at a large distributed system. Wow! The probability that, let's say one processor encounters some sort of issue as it's executed may be very small. If you take a large distributed computation, remember tens of thousands, hundreds of thousands of cores. The probability that one gets something that hash becomes much higher, much, much higher.
Michael Carbin: And we started to see more attention. There's been a long held focus actually in the research community of thinking about these sorts of issues. We're starting to see these again, re-emerge the different flavor and different names and research that we've been seeing coming up the on Facebook over these years.
Michael Carbin: So how do we cope with this? I have a problem
Michael Carbin: practice and the type of errors I want to talk about here are trains dance non-determined scares in the sense that if I have a computation, I run it once,
Michael Carbin: and it encounters an error.
Michael Carbin: If I start over and run that exact same computation again on that same machine with high probability. I'm gonna get the correct answer.
Michael Carbin: So these are non-deterministic These are not permanent error. We're always going to get an incorrect with all.
Michael Carbin: Well, in this instance,
Michael Carbin: classically, what we've seen is people use something like replication,
Michael Carbin: right? So there's been a long history of essentially taking a computation that I might have, and I duplicate it.
Michael Carbin: I ran twice. I check to see if the results agree.
Michael Carbin: The results disagree. Then i'm going to go ahead and run it again. I'll keep running it and running it until I get the right result. Of course, there's some specific restrictions on the types of errors and frequency of errors in which there is a sensible approach.
Michael Carbin: This has been a class approach. It's so a classic that going back in the computing literature you can find that the first time this really came into computing was von Neumann
Michael Carbin: when he was trying to construct reliable computations from these unreliable components, and what was very interesting at that time. As for if you go read this paper, it's actually these unreliable owners look a lot like neural networks, and I guess i'll come back to that.
Michael Carbin: But I think it's It's reminiscent of again these types of concerns that we're doing within model of software where things like neural networks are starting to become part and parcel of the way that we're.
Michael Carbin: So now replication plastic strategy for doing this. But I want to encourage us to think differently about this problem strictly. How might we solve this otherwise?
Michael Carbin: Well, one alternative approach here is what if we just let errors happen right instead of trying to correct errors. Let's just let's just let them have let's just let system process, and what what what could go wrong?
Michael Carbin: Well, let's let's explore this right. I'm sure you know, if your so and said this to you, you probably got in your mind all these horrible ways in which this can go wrong. But I want to show an example.
Michael Carbin: And this example here is a very simple method for solving systems of linear occasions called the the Jacobi method, and you don't need to understand this program at all.
Michael Carbin: But I do want to highlight a few specific features of this. So one. This is an iterative. Algorithm So you can see there's this while loop where this program is going to run until it converges after some number of sense.
Michael Carbin: So that's the first one.
Michael Carbin: The other thing I want to highlight are these operations in red? So these are operations where I want to show you what happens if we inject errors into these operations, and let these errors happen, You let these errors let these operations be corrupted, and let them compute on and go on and see what happens.
Michael Carbin: So this is what happens. So here is an illustration of the Jacobi method executing over some number of iterations. So iterations of x-axis. Why, what we're going to have is the difference between our current. Yes, it's a solution to the system linear equations, and in the actual solution.
Michael Carbin: And so this blue line is, if I just run the computation with no errors. Let me see this nice convergent behavior over time, which is very nice.
Michael Carbin: Now, this red line here is what happened if we let errors happen,
Michael Carbin: And in particular we're going to inject errors here on iteration, ten thousand.
Michael Carbin: And So what we see here is initially, there's quite a significant leap up
Michael Carbin: right in error from where we were before, and in fact, it looks like we've lost nearly five hundred or six thousand iterations of progress
Michael Carbin: by encountering this error.
Michael Carbin: However,
Michael Carbin: if we actually let this computation continue running out, what we see is that we actually add only two point, six percent more iterations until we converge
Michael Carbin: to an equally accurate result. So this is interesting. This is interesting, because now it gives us another strategy for thinking about dealing with these types of errors. Right? So, on one hand, we can do something like replication. And so replication. You might think this is be even horribly expensive, and in some cases it really is. But if you scour through the literature, and they're
Michael Carbin: very clever tricks for compilation and hardware support, you can find that maybe it is down to fifty percent in this, and at least in latency energy. Harvard, All these other things. There are other measure that you may need to keep, in fact,
Michael Carbin: conceptually fifty percent open. And we're thinking about time to execution.
Michael Carbin: Now, approximation. On the other hand,
Michael Carbin: what this gives us is instead two point six percent overhead.
Michael Carbin: But the difference Now, the difference here on the crux is that our intermediate results or our results in general may differ from what we intended them to be by some epsilon.
Michael Carbin: And so this Epsilon is important.
Michael Carbin: So when I think about uncertain computations. What I think about our computations where we've composed the software system, and somewhere along the way, or maybe even at the output, results are being computed to some imprecision or inaccuracy,
Michael Carbin: and classically as computer scientists, we have this very discrete view of the world. We don't necessarily think about these things. Of course, if we talk to our friends in numerical analysis or scientific computing is something they think about over time and over the course of time. What I found through. My research is if we can move towards software systems
Michael Carbin: where this epsilon is more explicit,
Michael Carbin: we get new opportunities. And so we, seeing one right here with approximation. Or if we know about this Epsilon, we can reason about this epsilon. Perhaps we can build new systems that are more efficient and tolerant of things like hardware errors.
Michael Carbin: And I think this is critically important, as we're thinking forward as a community of how computing is going to evolve, because this type of uncertainty is now absolutely everywhere.
Michael Carbin: So it's coming up from below from from our hardware systems. We already see this with things like errors. Um that are happening in our hardware systems. But even more you think about the trajectory of machine learning system. They're moving towards reduced precisionances where we're going to have approximation of numerical computations that we're doing.
Michael Carbin: We're seeing very emerging computations in stochastic or analog or neuromorphic computer, or even quantum review. But there are suddenly these fundamentally probabilistic or uncertain computational substrates that we can use. We need to use,
Michael Carbin: or the trade-off is we can get much more efficient or much more capable systems,
Michael Carbin: but it's unavoidable that we now need to think about the subsequent
Michael Carbin: the other way. This is coming up is from the application, or more specifically thinking about the environment, the fact that we're building these applications that are trying to reach out and model the fundamentally uncertain physical world.
Michael Carbin: So take, for example, this computer vision example here, or something like object detection. The goal of these types of systems is to try to actually identify the objects or the locations and positions and boundary boxes around objects that we have in the stream. And unfortunately, this system only has noisy axis,
Michael Carbin: right to this physical world, right? This environment is only partially observable. The data that we're getting back from sensors is noisy, and the models that we're using to construct these types of bounding boxes in essence are are fundamentally only correct with some some sort of probability, because they're typically approximations
Michael Carbin: and as well moving across to other applications and machine learning, image processing computer vision quantum. Again, we have fundamentally applications themselves, for this Epsilon is now
Michael Carbin: so uncertainty These epsilons are starting to show up everywhere in our computing stack
Michael Carbin: again come from below coming from above. And this is interesting. It's amazing. It's giving us new capabilities. We're able to build software systems that press. We couldn't even build before
Michael Carbin: by thinking about what neural networks are really able to do these days and processing on with certain sensor data that's coming in. We're getting very interesting opportunities for performance, Thinking about again what happened with approximate computing or this approximation opportunity. I talked about a few slides ago,
Michael Carbin: and of course, resilience as well resilience to errors if things go wrong, and they're only epsilon wrong. Can we figure out a way to still work through those types of there. But there's a fundamental research question here which is well, how do we build
Michael Carbin: software this way?
Michael Carbin: How can we productively build software. That has all the nice properties that we think about things like modularity and compositionality, and things that we can reason about in the presence of these Epsilons, because this is not how we formally or traditionally constructed software. It Hasn't been the main focus computer science again, been thinking about deterministic and discrete.
Michael Carbin: So that gives me brings me essentially to a world really the subject of of today's talk. And really what I've been doing over the years of thinking about ways of building software in this new reality from programming methodologies. How do we think about these new emerging software systems that are now mixed between classical code
Michael Carbin: neural networks? And now, with things like Chat, Gbt, Gb. Four. All the other models that are out there, perhaps natural language, that there's really interesting compositions where ultimately, at the end of the day. Even what these computations do. We only know what these some of these computations do up to, sometimes
Michael Carbin: moving beyond that their fundamental questions in semantics, or many of these computations now are real valued, but nondeterministic or probabilistic. As I said before, this is this is not where computer science has spent most of the time right? And instead, we need to fundamentally think about what are the semantics of programs that actually execute with these types of data types, because It is actually fundamentally different
Michael Carbin: beneath that. When you think about how do we represent things like real numbers, non-determinism, probability How do we compute what are efficient representations of these types of data types, so that we can compute on them productively in these types of applications.
Michael Carbin: And then, finally, reason,
Michael Carbin: once I have this software system.
Michael Carbin: How do I actually now reason about what sort of guarantees that I can get from this? And I actually prove that with reasonable probabilities can satisfy some specification that I have in my mind. Or perhaps isn't going to be epsilon close to the specification I have.
Michael Carbin: So today, what I want to do is I want to talk to you, maybe just a couple of these. But I'm going to motivate this. You know on how I've been seeing this. I've gone about looking at this problem from a very specific application
Michael Carbin: of thinking about how do we build software sort of in this domain? And let's say these programming methodologies that are very rich and expressive in the sense that we get networks and perhaps other specific techniques really took them to solve classic problem, and it's one of my great problems in in in compilation. How can we build new compilers that incorporate learning?
Michael Carbin: Ah! To tackle
Michael Carbin: the fact that now our underlying uncertain. Our underlying hardware systems are actually very uncertain when we need very expressive models on rich, learned models for behavior. So i'm going to do that, and then use that as an example, really to concretize a few of the opportunities that I see in this space.
Michael Carbin: So
Michael Carbin: the core a compiler. So this is a an incredibly simplified perspective from. I know we have a broad audience here
Michael Carbin: for some of you. You spend time hacking on compiler. I try to write a pilot every year. It's sort of my therapy. We're all right. But for many of you this is perhaps a reasonably simple diagram in the sense that I have source, code, c. Plus c. Java, something coming in on one end into the system, and then on the other side. I'm going to get machine code,
Michael Carbin: and of course they shouldn't be broken out into finer and finer granularities. If you actually know how these systems are composed, which is oftentimes the front end, going from source code to an intermediate representation, and then from that intermediate representation, then down to the machine code itself.
Michael Carbin: Now, where I want to focus. My attention is mostly on this this back end, where we have this rich intermediate representation that represents the program.
Michael Carbin: And we're figuring out, How do we take that.
Michael Carbin: I really map it onto the machine.
Michael Carbin: And so this problem, this optimization problem we can think of
Michael Carbin: as follows: So here is a simple example
Michael Carbin: where I have this code here in the middle, where I've got three little instructions and my goal here. One particular goal here that you can have in a back end is, we want to figure out what's the optimal ordering of these instructions so that I can get the best performance on the underlying.
Michael Carbin: Now. So this this border, what is an overall? It's going to be some permutation of the instructions that I have, and this is going to give us essentially a transformation space. So the in space of valid transformations are going to have of this particular code, and we can represent that here as this as this dag that i'm showing here.
Michael Carbin: But what this is representing is the data dependencies.
Michael Carbin: Obviously I can't compute, C until I've computed A and B. So A and B. Need to proceed, c. In any particular ordering that I have of this computation, that I'm. Ultimately going to generate
Michael Carbin: So any topological sort through this graph is actually be a valid ordering thing in have of a conversation.
Michael Carbin: So that's my transformation space.
Michael Carbin: The next total component here is I have something that's called a performance model. So I want to take any permutation of these instructions, and then be able to predict what the underlying performance is going to be on the actual machine,
Michael Carbin: and oftentimes what I want to be able to protect is because I want to be able to explore broadly in this transformation space the transcription. Space may be much larger, just the two options that we see here. Rather it could be thousands, more than thousands exponentially, many even in the size of the program. And with that in mind we want efficient ways to be able to search this rather than actually trying to run perhaps a potentially expensive competition on an online machine,
Michael Carbin: right? So our performance model is here to actually help us out. So, for example, here's some
Michael Carbin: examples where a performance model may say, Okay, the cost of this one is going to be three. There's first one that costs the second one. It's going to be two.
So now, with that performance, model in hand. Now we can search.
Michael Carbin: There's a methodology for trying to search efficiently. Search these types of spaces, and this particular instance we're going to go ahead and choose this one with cost two which we think is going to be more efficient. So this compiler back in this is
Michael Carbin: in some sense the basic strategy that we have. We're now a critical component for getting to this work is is the underlying performance model.
Michael Carbin: How do we understand how to model what the machine is doing beneath us, so that our compiler can make effective search decisions and how to generate.
Michael Carbin: Now, this performs modeling problems
Michael Carbin: always been a very tricky problem. It's becoming even even more tricky in in modern computing, because not only are we thinking about this, our traditional classic processor, we're starting to get these very heterogeneous.
Michael Carbin: They are spanning many different chips,
Michael Carbin: very divergent styles. Some are very efficient in particular dimensions of particular computations; some really want to use for very expensive computations, you know, such as machine learning, and we really want to be able to map essentially an application in parts of that application down
Michael Carbin: to each of the individual chips that I may have available inside of my system. And so this is the sort of heterogeneous mapping problem is is a heart problem, and it's become increasingly hard.
Michael Carbin: And so in the systems communities there is just this fundamentally hard systems challenge that many people in the community at this point are trying to have, which is, we want to take our program. We have some sort of hardware specification of all the components that we may have the sea of components that we may have down here at the bottom, that we may be targeting, and we need a performance model,
Michael Carbin: and our compiler is supposed to. Then, you know, magically come in, navigate this space and generate code that appropriately targets the machines below.
Michael Carbin: Now this is becoming increasingly harder,
Michael Carbin: partly because what have I talked about before in the sense that each of these components, each of these
Michael Carbin: chips, each of these machine components, may have a different behavioral model. You know. Again, we're moving away from these traditional models of computation where I have single input going into my process, learning in a single input that's coming out. And instead, I need to think about emerging models of computation
Michael Carbin: or single inputs may now be turning into distributions of inputs
Michael Carbin: and individual devices may have very different distributions of inputs. And so now, if I really want to think about their emerging systems challenge, I need to broaden that to now include this behavioral.
Michael Carbin: And so given the complexity of these,
Michael Carbin: and the fact that really the only effective way that we can measure
Michael Carbin: the true behavior and performances of machines, as often through observations as refugees by running things and observations. This, in essence is
Michael Carbin: has for a long time been viewed as a learning problem, and is increasingly so getting viewed as a learning problem in something where we're starting to see very effective techniques for applying.
So what I want to talk about today is a little bit of our work that we've been done on the performance modeling side
Michael Carbin: of using machine learning to build these performance models right and again, with an eye for concretizing exactly the
Michael Carbin: how these sorts of exotic software systems are starting to be developed again, being motivated by the fact that well learning may be one of the most effective ways to capture the underlying uncertainty, the exact behavior of the underlying machine. And therefore, if we incorporate learning, we can get a more effective
Michael Carbin: so performance one.
Michael Carbin: So we already got a little peek preview of this before a few slides ago, and as I mentioned. This is critical for compilation, but also other areas of forms, engineering and debugging. But essentially what this is going to take is
Michael Carbin: some piece of code. Let's say X, eighty, six basic blocks. This is the domain in which we've actually validated some of our results, and we can throw it into a performance model, and it's going to give us a number of cost. And here, in this case, what we are looking at is throughput, so throughput
Michael Carbin: of sequence with structures is essentially, or to think about taking this code and just unrolling it all the way to
Michael Carbin: right and then processing it. It is in the steady state of executing that.
Michael Carbin: How many cycles does it take to execute essentially that basic block, an iteration of that basic bar? And again, thinking about the steady state unrolled and averaged off across, perhaps an infinite number of iterations,
Michael Carbin: and you know the lower. This cost the fewer cycles it takes the better or the faster I can execute that code. So
Michael Carbin: now this problem is hard and alluded to. It's hard, and I want to give you some more evidence that it's hard to go through some examples here. But the reason it's hard is because this is a very complicated device, a system that we're trying to characterize. So if you look at your modern microprocessor, what you're going to see is
Michael Carbin: is a very sophisticated type of system, and if you're building a performance molecule, an accurate performance model. You need to figure out or come up with some effective model of many of the components of this system where instructions are coming in in this front end.
Michael Carbin: They're being fetched. They're being decoded. They're being scheduled. They're being actually executed. There's also, of course, a whole memory subsystem that has to be dealt with. And if you actually, in principle want to model this, you need to come up with essentially some model of the behavior of all these individual components, how they work, how they fit together, and of course, the appropriate cost
Michael Carbin: associated with each one of these.
Michael Carbin: Now, what makes this more challenging is while we had these sort of cartoon,
and there's some fidelity that we have in these diagrams.
Michael Carbin: The reality is that the true access that you're going to have to. One of these machines is going to look like this
Michael Carbin: right? And that's because
Michael Carbin: we don't have a full specification.
We do have some help.
Michael Carbin: So the Intel. And so when I give this part of a talk in in person, there's usually someone that sees the Intel manual slides and sort of puts their head down and starts crying, weeping, because they've spent endless hours reading through this documentation with trying to figure out what exactly your process is coming,
Michael Carbin: but it's it's a beast, it's, it's, it's a beast. It documents many, many aspects
Michael Carbin: of of the architecture, interaction, specification of optimization, cpu parameters things that can help you towards the end of building, if not a mental performance model, an actual physical performance model.
Michael Carbin: And again, this is this is filled. So it's filled with things like these sorts of instruction tables.
Michael Carbin: But again, this is not
Michael Carbin: it's.
Michael Carbin: Now, if you actually looked at
Michael Carbin: these instruction tables and you go through what you start to see or are footnotes like this. So these instruction tables are telling you various aspects of the latency throughput of instructions on on a given machine. But if you start to dig in here you'll see comments like this: All tables in this appendix.
Michael Carbin: Our estimates
Michael Carbin: right. Actual performance of these instructions right? Can range from somewhat faster to significantly faster. So what it means as though you're getting again a peak. You're not getting a full peak,
Michael Carbin: and partly because this is a very hard problem. So very. These are very complicated systems, and two Intel's credits are trying to do a very reasonable job of trying to expose some sort of specification that people can work with. But
Michael Carbin: this abstraction, this single instruction, abstraction with these latencies and throughput is is the
Michael Carbin: complete abstraction. It's and it's very hard to try to deliver this this abstraction.
Michael Carbin: So what this means is if someone's sitting down trying to build this performance model. Oftentimes, of course they'll be consulting these tables. But within the full complexity of this process, what they'll do is they'll start to make some kitchen, especially when you're thinking about compilation rather than you know other areas of research when you need higher for that.
Michael Carbin: And so what they do is they're abstract.
Michael Carbin: They'll choose a smaller subset of these components make some assumptions about their behaviors, and then set some parameters based on what they found, and they're going to get this.
Michael Carbin: And so Now your modern performance model, at least, is represented in something like your modern compiler. Llvm. Is going to look something like this very simplified version of the reality that we have
Michael Carbin: now is an example here just to see a hard. The citizen
Michael Carbin: i'm giving you a test. No, I don't expect i'm an actor.
Michael Carbin: We're gonna walk through this one. So here's an example, and that is how many cycles does it take to compute one hundred iterations of this following?
Michael Carbin: Now I don't expect you to understand instruction, but we'll go ahead with the Intel Manual, and we'll go ahead and decode this instruction. So I go and look at this page and the Intel Manual. What i'll find is Aha! This is actually it's it's a Vector instruction. But what it's doing is essentially a bit-wise logical Xor
Michael Carbin: of these registers,
Michael Carbin: so I can rewrite this. And now I get something that's maybe a little bit more interpretable to your your average computer scientist, or I've got some register r one, and i'm going to
Michael Carbin: store into it the value of R. One X over two,
Michael Carbin: and now I can start to try to figure out a performance model. It can go to another part of the Intel Manual and look up the latency and the throughput for this instruction again similar to those tables that we saw a few slides ago.
Michael Carbin: So if I go and look here, what I can find is that destruction has a latency of one. So what that means is it's going to take one cycle. The abstraction is, Make one cycle, the processor to execute this instruction, and then it has a throughput
Michael Carbin: point three three. You are really one over three. And the way to think about this is
Michael Carbin: it is a processor
Michael Carbin: can execute three concurrent instances of this instruction at the same time, provided there are no data dependencies between that. So an excuse
Michael Carbin: breathe is the same time,
Michael Carbin: and he's a numbers before.
Michael Carbin: So now, with this information, we can take a first crack of figuring out. What is it? How many cycles will it take to compute one hundred iterations.
Michael Carbin: I Oh, this instruction!
Michael Carbin: It's!
Michael Carbin: There are a wider idea of different answers when I better do this in person,
Michael Carbin: some all of them actually pretty good, because this is, this is a weird part.
Michael Carbin: But so the first one and very reasonable one here is actually the answer that you would get from Llvm's performance model, as of several years ago. When we were doing this work, and then, after they saw our work, they they changed performance. But
Michael Carbin: this is the answer, they get one hundred cycles. And the reason why you would think that is because, while I've got this instruction here,
Michael Carbin: it takes one cycle, and, Moreover, this instruction reads from R One: The
Michael Carbin: then writes to our right: I can't begin executing my next instruction
Michael Carbin: unless I've figured, or unless I've completed executing the previous instruction if they're reading and writing from that that same data point.
Michael Carbin: So this is. This is a very reasonable thing to start with, just by looking at the data dependencies, right? So our one is going to flow in R one. So
Michael Carbin: of course, this is going to take one hundred cycles, because each instruction takes one cycle to execute, and this data depends between.
Michael Carbin: Now There, there's a trick here, right? So if you, if you look at this a little bit more carefully, or and perhaps you're familiar with your bit twilling and bit vector arithmetic. What you'll see is that Well, this is actually this: is this Isn't: Really, this instruction, this is really this instruction. I'm: setting R: one to zero.
Michael Carbin: You're like, Okay,
Michael Carbin: that's interesting, like, Why, this is a cool optimization like um. I feel like my compiler should generate that. Well, not only might a compiler try to generate that your processor actually I dynamically identify this
Michael Carbin: rewrite the instructions. So if you go and actually point around through the Intel, you
Michael Carbin: and you'll find that there are zero idioms. Well, the processor itself has been manually optimized to find these sorts of zero idioms. And if you go and find this list, you'll see our instruction that we have. That's our mnemonic that we were looking with before we we wrote it to be simpler,
Michael Carbin: and what we'll find is the processor finds all instances of these zero idioms, and they get removed, and they have no execution, and so great. Suddenly our latency goes to zero,
Michael Carbin: and now we can, instead start thinking about throughput right. There's no dated prints here and throughput here said We can execute three at a time.
Michael Carbin: So what that means is now. Well, maybe this is going to be thirty three cycles.
Michael Carbin: That's it bigger.
Michael Carbin: But
Michael Carbin: you do some more poking, and you actually go and measure this, which you end up with this twenty, five cycles, or if you use Intel's proprietary performance, model performance analysis tool by Aaka that they had released at the time. You see this twenty five cycles, and then you start to wonder what's what's going on here.
Michael Carbin: And, in fact, what eventually ends up happening is once you go through this renaming process, You're no longer even bottlenecked by these instruction specifications, and instead, you become bottleneck by the front end, which can actually do four instructions for cycles. And so you get an entirely different
Michael Carbin: ultimately when she published this.
Michael Carbin: So this is this is hard.
Michael Carbin: This is hard, because
Michael Carbin: how do we go from what Intel gave us?
Michael Carbin: It's this very reasonable instruction specification which you know an actual production compiler was using, because this was the process that they use to the actual measured performance or the proprietary performance. When Intel has worked in all the optimizations and smarts,
Michael Carbin: how how can we cobble together all this documentation, our understanding and measurements, to get there?
Michael Carbin: Well, in practice. What people do is they do this. They will build their simplified performance, model the
Michael Carbin: they will stack that against their actual processor They'll get a data set of examples.
Michael Carbin: I'll pass that through. Both.
Michael Carbin: Compare the difference. See where they're wrong.
Michael Carbin: This by and large. If you go and look through the development process of many of these certainly open source
Michael Carbin: performance models. You'll see this iter of refinement process.
Michael Carbin: And so in the process. When we were doing this research at the time. We sort of looked at this and said, This looks an awful lot like machine learning
Michael Carbin: good data set. I've got labels. There's some sort of model. I'm trying to tune. I can look at the difference, and i'm refining my model. And so why Don't, we just go ahead and replace that with the neural network. Let's let's just see what happens.
Michael Carbin: And so on. Some work called Iphone. All this is exactly what we did, right where we came up with the neuron at work. We built a data set, a very extensive data set at the time that had some specific invariance designed to capture and variance that we have for these performance models. Things like no cache misses preemptions,
Michael Carbin: very simplified, setting not the full month thing, but this is, in fact, how systems like Llvm's Yay, they actually do model themselves.
Michael Carbin: And then, of course, we get a comprehensive dataset.
Michael Carbin: And so we went and ran this, and we found that we can actually using this technique Just moral neural networks build a system that had half of the error of Llvm in into as Aka. No, at that time.
Michael Carbin: And so this this was great. This was an amazing success. I think it was one of the first papers really out there, at least in the modern era to be able to demonstrate that we could scale up this type of deep learning to actually target this type of problem.
Michael Carbin: And so I them all. There are some details to its architecture. I don't really want to focus on those too much. There's some novelties, but I think the most important part here is really that it's entirely an unrepresentation
Michael Carbin: so unlike previous work and incorporating machine learning into these types of recording volunteering situations. What I from all did is, it takes text, tokenized text and predicts a number.
Michael Carbin: There's no featureization of the code. We don't need to extract related penses. All these sorts of things.
Michael Carbin: Nothing right. Learned representations has has been the interest and fair of, Let's say, the past,
Michael Carbin: you know ten years of modern deep learning, which is that we can move away from the explicit futurization of our data and have these incredibly capable systems come in and learn purely from inputs or text
Michael Carbin: to get a very effective result.
Michael Carbin: And so, if you look at it, if you come back,
Michael Carbin: this example that we have, if Mall ends up getting pretty close to this measured version. So it actually is able to figure out this optimization that is happening beneath the scenes by the processor that's been detailed truly from date purely from from yesterday.
Michael Carbin: Now, so I them. All this has been great. It's taking on our standard methodology. We
Michael Carbin: replaced it with a neural network.
Bye,
Michael Carbin: but not all is perfect
Michael Carbin: right? Because one benefit of your traditional performance model is, you can in principle understand what's happening. These These performance models oftentimes when they're written, they sort of have a trace to some sort of explanation. There's a There's a simulation
Michael Carbin: that you can read off that models individual stages of the processor that gives you a hypothesis on how the system believes the underlying processor is actually behaving. And so this can be very helpful for things like performance to. I really trying to figure out. How do I optimize my system, change my code
Michael Carbin: based on the observations that i'm staying from the simulation.
Michael Carbin: So there's lots of interpretability motivated us to think a little bit differently here,
Michael Carbin: once we had I them all, which was, you know. Can we maybe try to combine the best of both worlds, where, instead of entirely throwing out the
Michael Carbin: our analytical model and replacing with the neural network, maybe what we can do is
Michael Carbin: a combining both and more of a neural symbolic approach.
Michael Carbin: And so in the next line of work called Diff tune, we did exactly that, and the way that we started out is, we said, Okay, let's look at this llvm. Um um Mca. Simul that we have, which had a very detailed simulation pipeline,
Michael Carbin: but unfortunately
Michael Carbin: had some wrong intuition on what was actually going on.
Michael Carbin: In particular. It's parametrized. It's parametrized by these sorts of numbers that we're seeing down here at the bottom, and it has its own hypothesis on what these numbers are, and we said, could we use machine learning to delivery, figure out better sets of parameters to then plug into Llvm: Yeah. To, therefore, get better accuracy, but still
Michael Carbin: be able to interpret essentially a trace of the behavior of the underlying system.
Michael Carbin: So we dug into a l of themca, and, like I said, it is a It's a fairly detailed simulation again. This isn't going to be, you know, architecture or level research
Michael Carbin: psychoactive assimilation. But it's reasonable enough, as is used inside of a compiler to be able to validate various scheduling-based optimizations that are going on inside of
Michael Carbin: now. L. O. Mmca is is not small. It's not super massive, but not small. It's about ten thousand ten thousand lines a Cpos,
Michael Carbin: but where it does become big, is. It has about eleven thousand integer-valued parameters
Michael Carbin: that determine its behavior.
Michael Carbin: And so this this is the challenge and these parameters come from it trying to plug in the pieces of this model that it has right? So does the dispatcher
Michael Carbin: right, which essentially is a very simplified model of of a front-end instructions coming in. At what rate did they get depoted and dispatched up to the rest of the processing pipeline, and was single prime.
Michael Carbin: How many instructions can be dispatched per cycle.
Michael Carbin: There's also reorder buff, right? So it's also modeling out of order execution and the
Michael Carbin: the size of the reorder buffer. Right. There's some number of instructions that it has.
Michael Carbin: But where things start to get more complicated is that it starts to have per instruction Parameters such as these right latency and meat advice and read advanced cycles, which is, if I have instruction, and, for example, it has a right. How soon can that
Michael Carbin: computation be made available to a subsequent instruction before it actually really needs to be fully retired
Michael Carbin: and as well. There are execution units with things like Port Mac, so I don't expect everyone really to understand all the the details in here. But the real primary point here is that every single one of these individual components we have some manner of parameters, some of them global, some of them being very specific to the individual instructions that we have,
Michael Carbin: we're all untold together. We end up with eleven thousand parameters
Michael Carbin: where
Michael Carbin: the low Vmca. Developers. Unfortunately, the way they do this is how we've talked about spending a lot of time looking at design documents, and also doing some measurement and refining.
Michael Carbin: So with our system d tune.
Michael Carbin: When we wanted to apply learning, we thought about this really as an optimization problem
Michael Carbin: where the way we can think about this is,
Michael Carbin: I have some simulator, right? So F is my simulator here, right? It has some set of parameters data, and on a given input X. It's going to produce
Michael Carbin: protection, right? Some some model the performance of that particular input meaning a basic block,
Michael Carbin: all right. How many cycles it's going to take.
Michael Carbin: Now, the optimization problem is Well, if I have a data set right,
Michael Carbin: can I essentially minimize or find the best set of parameters that minimize the discrepancy between this prediction that I have, and a true measured ground-truth prediction that I have for the data. Set
So nothing Fancy here,
Michael Carbin: right? This is just optimization problem, specification of thinking about. How can we
Michael Carbin: optimize the simulator or it's indexed again by about these eleven thousand parameters, which together give us an absolutely massive number of possible configurations.
Michael Carbin: So this problem, as it's posed is unfortunately horribly interesting. Um, using search or stochastic search as we validated by using other tools available in the techniques, such as open tuner um available in the community, such as open tuner, These existing tools that we have just simply do not scale
Michael Carbin: to this type of problem.
Michael Carbin: So what we said is perhaps we can build a surrogate optimization problem.
Michael Carbin: So the key thing here is that this is exactly the same optimization problem except you have an F hat here. Now,
Michael Carbin: where the idea is, F hat is going to be a circuit.
Michael Carbin: It's going to be a neural network that approximates the behavior of the simulator.
Michael Carbin: No conditioned on a center parameters right. So given some set of parameters. Theta This surrogate is going to intuit what our simulator would do
Michael Carbin: given those parameters. Theta
Michael Carbin: And the reason why we did this, and with this approach is because now it's going to allow us to do gradient-based optimization. What we can do is now take gradients with respect to these parameters.
Michael Carbin: Some judges gradient descent to try to figure out if we can find effective parameters. They're going to minimize
Michael Carbin: this this problem that we have.
Michael Carbin: And just to put this on pictorially what we're trying to do here
Michael Carbin: is this is sort of a visualization of
Michael Carbin: in an instruction that we're going to have inside of our of our architecture, where there's a parameter that we can control, particularly the dispatch with
Michael Carbin: right. And this parameter of llamca, we can look at the timing or the latency of this particular, we look at the throughput of this particular instruction as a function of this parameter,
Michael Carbin: and so what we can see is Lvmca. What it's going to do is for each individual discrete point of this parameter. It will have a prediction.
Michael Carbin: Now, again, the problem is, this is going to be essentially a discrete optimization, and even using Zeroth order methods, we are unable to find effective ways to navigate to space. But if we build a surrogate,
Michael Carbin: What the Surrogate can do is now give us a smooth interpolation once trained between all of these points,
Michael Carbin: and so now, what it allows us to do is, try to find and navigate through this curve to identify the exact timing right where this is the ground truth timing that we're going to have for this instruction, and this is going to intersect sleep between here and here,
Michael Carbin: and this is going to give us some understanding of what this parameter should be, which we can now find. Your gradient descent
Michael Carbin: is our surrogate optimization problem.
We're gonna have this f hat instead of our traditional simulator, our discrete simulator, which is otherwise
Michael Carbin: not differentiable. This is a highly discrete, highly discrete computation that has not only discrete integers that it's operating it, but very many discrete data structures that it's using to organize the simulation. So this neural network gives us now a very differentiable surrogate of this otherwise discrete contradiction.
Michael Carbin: Now, where's the surrogate? Come from. Well, what we did is we use? I am all so. We used our previous approach. If the Mall changed it ever so slightly, so it can be conditioned on a set of parameters. We used a variant, an iphomol architecture to then mimic the behavior of our surrogate. So now this is going to introduce a surrogate learning problem.
Michael Carbin: Our goal here is now to find a hat
Michael Carbin: again over this data set that we're going to have. This is a data set we're using for learning slightly different, enumerating over this data set over the
Michael Carbin: parameters and also examples. What we want to do is minimize the difference between our surrogate here
Michael Carbin: it are actual, simply
Michael Carbin: so. We are training our surrogate to exactly mimic
Michael Carbin: and the best that it can. The behavior of our simply so over a wide variety of different parameter settings.
Michael Carbin: So the result here
Michael Carbin: are, or actually quite great, so compared to open tuner. Again, using essentially at that time the state of the art that we had for doing Zeroth order methods. It uses a wide portfolio of random search techniques and zero other methods to to try to find solutions to tuning problems for for systems
Michael Carbin: and so on. The left. Here. What we've got is the mean, absolute percentage error. So lower is going to be better here and here. This is me validated on various different micro-structures that we're going to have from intel and also eighteen. And so what we see is open intruder by far unsuccessful in this problem.
Michael Carbin: Our experts, these were the default expert parameters that had been provided from us, and it had been tuned manually over the years through this open source effort. And then, finally, we had. We had this tune, which came in essentially all instances with either competitive or or better accuracy, and this is again learned entirely from data.
Michael Carbin: So we go back to our example again.
Michael Carbin: No, L. Ovm. Where we started at was one hundred. We had it them all, which did a great job of really getting much closer to the true measured performance, and even again expert, modeled, proprietary expert model performance,
Michael Carbin: and then dive. Tune comes in at twenty seven. So it's actually able to come in and get very close to our at the moment.
Michael Carbin: But again we're covering the interpretability now of Llvm, in the sense that we can get full trace of what we anticipate, or what Lvm Nca. Hallucinates, or believes is the underlying execution behavior of a piece of code on our march.
Michael Carbin: So now
Michael Carbin: I them all d to, and even efforts, that we've had continuing on from there have been tackling this performance, modeling problem learning.
Michael Carbin: We've We've had some work in behavioral modeling coming from a variety of different directions, and oftentimes from our approximate computing. Mark. I'm not going to talk about that there. But we've also made efforts in the past several years to think differently about even learning the compiler and the compiler itself can now take
Michael Carbin: It's different. Or this he's learned performance models. These learn behavioral models. And really now, can we think about automatically generating compiler or automatically generating a back-end.
Michael Carbin: Can we do all of this really from first principles from data?
Michael Carbin: Without necessarily the the manual process that we normally have Here and there's been a very vibrant community standing up doing work in this space right now. But there is by and large a of a grand challenge here of automatically synthesizing a compiler from data. And of course, a hardware
Michael Carbin: now. But what I want to highlight here again is the process that we went through.
Michael Carbin: So we had this standard development methodology for a performance modeling the
Michael Carbin: um. We then realized that what we're trying to do is it's a modeling problem, right? And in particular, this is we're trying to model uncertainty. We have some fundamental uncertainty about this this hardware, substrate, and neural networks. In the last decade or so have emerged as a very effective technique for modeling uncertainty.
Michael Carbin: So that's how we moved to I
Michael Carbin: now, of course, with item all we lacked. Now the interpretability of understanding our system. While this may be great, this does, while it may be great in performance in terms of how well the model matches reality.
Michael Carbin: It's, unfortunately, not necessarily usable for all circumstances, because in many circumstances we do want to understand what our system is doing,
Michael Carbin: and So with that we move to div tune, which is now this neural symbolic approach.
Michael Carbin: Now, of course, if you've been paying it, I imagine you've been paying attention to Jack Gbt and everything that's been going on in the world for the past few months of nine years. But you can see now this trend of what we have with software.
Michael Carbin: Again, where we were before neural networks, neural symbolic combinations of neural networks and traditionally written components. And now,
Michael Carbin: really a future of software that may even include large language models, instances where our program is now going to be half-code half-trained neural networks. We're not half a third code third trade neural networks, and maybe another third of just just English. Right? You're really starting to see people
Michael Carbin: start to report these very strange experiences of Yeah. My program is about half code and half English, and you prompts a large language model.
Michael Carbin: And this this is a very weird feature, right?
Michael Carbin: But I personally think this feature wasn't, man, before this is where we've been moving towards for some number of years.
Michael Carbin: Again, as I've said before, uncertainty has been creeping in downwards from our environment,
Michael Carbin: thinking about how we're deploying systems on all these new places where we're trying to get them to model uncertainty and machine learning has finally emerged as an effective way to model a certainty really end-to-end large-scale, differentiable systems have really been you know a huge way for thinking about how to address many of these people,
Michael Carbin: and we're seeing that as well with things like Eisenhower and do
Michael Carbin: coming up below as I've talked about right. This is the other case where we're seeing uncertainty
Michael Carbin: and with tiff tun. Yeah, we're we're with iythm on diff tune. We are modeling this uncertainty in the hardware system as I've talked about right using these techniques where, of course, this hardware system is our environment. So this performance modeling particular system. But even moving forward, you know, as I talked about four, with the approximate computing example,
Michael Carbin: our hardware systems are just going to be increasingly less precise, right? And so here we see that with again this Jacobi iterative method where we saw. Well, maybe if we recognize that we're actually
Michael Carbin: fundamentally operating on a uncertain computing, substrate, and perhaps architect our software. With that in mind we can get performance benefits versus a replication traction.
Michael Carbin: Right? So for me, when I see a diagram where i'm being pushed from above and being pushed from below with uncertainty, coming in all these directions. I think it's it's meant to me that it was an inevitable that we're going to start thinking about
Michael Carbin: how we apply these sorts of neural symbolic systems systems themselves are starting to use uncertain components, things like large language models to really tackle the fact that they're all trying to model we're modeling this type of uncertainty in our in our modern software or where we're deploying our modern software systems
Michael Carbin: really a first-order concern. And as we're seeing it just can no longer be more.
Michael Carbin: And I think personally, there's a wide variety of different interesting research directions to take that we've been discovering, and I think that many people in the community have been working on as well, and I hope people are inspired, perhaps today to go and think about some of these problems as well, but just as a simple preview of some of the things that we've been doing.
Michael Carbin: For example, we've been thinking about what what is a programming methodology. Now we think of methodologies like you
Michael Carbin: declared a program or functional program,
our object-oriented program
Michael Carbin: like, What what are the types of programming methodologies that make sense in this in the space? And one thing that we've had in the previous few years inspired by what we've done in tune is actually a surrogate program which is starting to outline techniques and tactics for thinking about building these types of surrogates, such as what we saw with fifty.
Michael Carbin: So, for example, the circuit, just as we we had talked about the Surrogate is going to be a neural network that's going to model the I o behavior of some.
Michael Carbin: And what we've wanted to work towards is, can we start thinking of a theory of learning surrogates? And maybe we've been associated program analysis that allow us to write a program that we're trying to surrogatize for some definition to the problem that we're trying to or to the model that we are.
Michael Carbin: We've made initial progress towards this right towards this theory of learning surrogates. So you know, programs are just functions. Is,
Michael Carbin: There's nothing super special about that. And the literature you actually comb through the machine learning literature. You'll find, for example, sample complexity bounds on the
Michael Carbin: a neural networks ability to learn classes of functions so correspondingly. There. There could be opportunities here to think about which classes of programs that neural networks can learn effectively if we're thinking about using surrogates as a primary programming tool,
Michael Carbin: so something that we've done over the past year or so has been using the intentional representations of the program. Since we
can analyze a program
Michael Carbin: and then use that to provide better learning bounds.
Michael Carbin: So, as a conqueror, it is concreteization here. If I want to learn a circuit of this type of program
Michael Carbin: where it has a branch in it like there's a large program inside. If there's a branch I can go left, or I can go right left to F and left to G right, and what I want to do is again learn a surrogate here.
Michael Carbin: But the question now is to learn a surrogate is, I need data.
Michael Carbin: What What data should I generate here? And clearly I need to generate data that's going to exercise both side of these branches. Should I generate this uniformly? Should I give away? Is there some sort of the input data distribution in which i'm going to deploy it. There. There are all sorts of strategies here, but this is going to be a fundamental question. If you're trying to build a surrogate for a system.
Michael Carbin: So what we found
Michael Carbin: that what we can do is actually do program analysis. We can try to use program analysis to reason about the complexity of the functions
Michael Carbin: below this branch, where so, for example, with a nice linear function over here versus something much more wiggly, which is going to have higher frequency. What you'll find is based on our analysis. Our analysis will tell you when you generate inputs. What you want to do is bias. Your examples more towards the complex programs. Complex part of the program rather than a simple part,
Michael Carbin: seems obvious in hindsight. But it's something where we can actually now start to build
Michael Carbin: programming languages, tool program analysis tools that now automatically inform the way in which we are going to train these sorts of neural networks
Michael Carbin: and in preliminary results,
Michael Carbin: where we applied this to a renderer, which is again, No. If you take diff tune, which is a no physical simulator of hardware systems as me, a physical simulator of the that amps of light. So we apply this to a renderer, and what we can see is
Michael Carbin: of our ground. Truth image here, and if we do a naive uniform sampling which you might otherwise do versus our guided sampling is maybe a much higher quality result for the same number.
Michael Carbin: The last one I wanted to talk about is really more open challenges. I myself have had work in the space of reasoning. How can we actually reason now about the correctness of software systems builds in these ways.
Michael Carbin: Um. But many people in the community have also as well.
Michael Carbin: And so I put this here. More it more is, you know, inspiration into a rope discussion, which is, How do we? What is? What does it mean for these systems to even be correct now?
Michael Carbin: Or certainly we have traditional properties like safety. Does a program satisfy basic invariance? Right? Does it always return a result that is greater than zero, right? It is a distance metric, and always needs to do this. No fire
Michael Carbin: do something strange. I approximate this program. It's a way. This is a basic property that needs to fold at the final property, or it could be something even more intentional where i'm. Never going to reference at all. These are classic types of invariance that we may want to be true of a piece of software.
Michael Carbin: Well, in this concept of uncertainty we may need to and not know what we do, we may need to. We really do need to rethink these types of properties, where now, perhaps, they're more probabilistic.
Michael Carbin: Safety hold with some probability, the
Michael Carbin: or are my results within epsilon of some idealized right that a history or results doing both of these things.
Michael Carbin: Now there are also concepts like generalization,
Michael Carbin: it starts to become new, You know. I think we've only informally thought about this in some with our work, in testing and even thinking about the difference between our dev environments and reduction. But
Michael Carbin: fundamentally, what we're What we're thinking about is a generalization problem. If I have one of these systems crafted in this way it's been developed. Now it's been fundamentally developed with with the data set that we've used during the training of this system. Will this system generalize in production to a distribution of unseen inputs.
Michael Carbin: Now, also, questions about robustness.
Michael Carbin: Do systems, behaviors change under slight perturbations, perturbations of data.
Michael Carbin: Now, if I were to ever so slightly regenerate my data set. Am I going to get an entirely different system with entirely different behaviors? Because that's not necessarily going to be the most comfortable idea in the world. I'm thinking about how robust our system may be to. For example, replication. We need to build it again. If someone is trying to replicate
Michael Carbin: the underlying system.
Michael Carbin: Interpretability, right? Can we actually understand and triage failures of these systems? What can we do? We
Michael Carbin: now? There's been a wide variety of different results going on the community of
Michael Carbin: all sorts of adversarial verification of neural networks, thinking about reinforcement, learning? How do you develop monitors and shields for reinforcement learning. I think a lot of this stuff is very interesting, and I think where this system or the
Michael Carbin: community broadly should go is how you start to really connect these to some of the the real problems people are trying to do in their day to day and constructing these types of software systems, and my hope is that we'll be able to make strong progress here, and maybe what it requires us is to rethink
Michael Carbin: how our systems behave a little bit differently, because, for example, with we go back to our Jacobiative method. So back in two thousand and eighteen, you know, we had this result. We demonstrated the benefits of the fact that we were only going to get two point, six percent more overhead. But we also sat down and wrote some proofs,
Michael Carbin: right? We actually realized that we actually bound the number of additional iterations that you may incur
Michael Carbin: by using this type of strategy for this particular problem. And so essentially, you have a delta T. That's going to be some function of the Epsilon that you're going to have.
Michael Carbin: How big of an area you're going to have.
Michael Carbin: And the reason why this worked is because these systems are self-stabilizing systems. So joel Kobe is very interesting. Where, if you encounter one of these errors, so Jacobi will converge to the right answer,
Michael Carbin: regardless from wherever you start. In the space of Guesses initial initializations, it's always going to converge. So that's good. And so, when you have an error,
Michael Carbin: what happens is
Michael Carbin: essentially, you just restart right. It's like you're restarting.
Michael Carbin: Now, fortunately, the way these errors propagate through these systems is not catastrophic. So rather than restarting from a random point. The reason why we're converging much more quickly is because we're restarting from a point
Michael Carbin: that is still preserved some of the information that we've made in these initial ten thousand iterations of progress. It's not truly a
Michael Carbin: right. So, starting to rethink our systems, as perhaps more of self-stabilizing systems systems where we can bound the impacts of errors. The impacts of uncertainty may lead us towards new directions and thinking about software.
Michael Carbin: So with that just going to wrap up my my personal opinion landscape the software is just fundamentally changing the
Michael Carbin: it's been changing, and I think it's It's It's finally here. I don't think you can deny the fact that virtually most programs are going to be written going forward, or now going to be programmed using a large language model which are generating code that
Michael Carbin: who knows who understands.
Michael Carbin: So I mean, this is the feature. It's
Michael Carbin: again future coming from below, new hardware substrates future coming from above, deploying these systems in very rich, expressive, partially observable environments. And again computations with these opaque machine learning systems.
Michael Carbin: Now the consequence of all of this is again that there's an Epsilon. There's these up along the way.
Michael Carbin: We really, as a community, need to start thinking more carefully about Epsilon correctness, probabilistic correctness. Because this is these are the tools that we need to think about what these systems do, and it's not a way. That, at least is piece of me as a classic computer scientist has been trained to really think about what our systems are doing.
Michael Carbin: But I think there's an opportunity,
Michael Carbin: if we may progress here. I think we can get much more amazing, much more capable systems, but systems that we'll also still be able to understand.
Michael Carbin: And so with that,
Michael Carbin: have you taken any questions in the Qa.
Dilma Da Silva: Thank you so much, Mike. This was, you know, an amazing
Dilma Da Silva: set up, or things for all of us to think about,
Michael Carbin: and really ah! And we have people in in in the Q. And A. Thank you for your talk, and ah, and and things like that I'll take. Ah! The first question that came early in your presentation, and I think that go ahead.
Michael Carbin: Can you see the the Q. And A. Or I can, I can.
Dilma Da Silva: I will. I will be, you know, triaging and asking you the questions.
Michael Carbin: Okay, yeah, go ahead. Let me see, I might be able to get there, but it might take a little time.
Michael Carbin: They just Yeah, that's okay. That's okay. So
Dilma Da Silva: So the the work that you started, you know, reminding us the A. Certain comes from everywhere, and then you start to talk about those applications and the Jacobite the ratio, as one example of applications that have some tolerance for faults that you know we had in the in the Q. And A. The comment that there was this direction of the and I think that this may refer to the working.
Dilma Da Silva: It's like supercomputing. Did you know they were because they worked so many with those models, right, those numerical analysis, iterative models. So they have been looking at that. Do you think this is coming from the same perspective, or how you may relate to when people thought twenty years ago about fault, tolerance, outreach.
Michael Carbin: Oh, no, absolutely. I do think this is coming from that same vein. Um! You know the work that we had there in two thousand and eighteen built on some of the observations that we're seeing in. So, for example, or systems like fault, tolerance
Michael Carbin: gymnasts that we're going on. These were again. Systems are coming from this intersection between the scientific computing, numerical analysts, mathematicians as well as a high performance. Hpc. Community of thinking about how we can build a fault, tolerant solvers,
Michael Carbin: and I think in that space there are some very interesting results that came out. I'm not sure how broadly they've been adopted since. But I I think my my take on these is really, how do we move that to other parts of her stack.
Michael Carbin: Right? How did we think about
Michael Carbin: fault, tolerance, robustness for more traditional applications as well? Where there's not necessarily quite a nice mathematical formulas?
Michael Carbin: We all need to start something, but I think that results. Those results of those times are very interesting and impactful, and I think that's the open question. Is, Can we generalize the observations there to other manners?
Dilma Da Silva: Thank you.
Dilma Da Silva: Now a question on
Dilma Da Silva: neural network correctness. So when you say that a neural net is correct with some probability. What does that even mean?
Dilma Da Silva: It is often the case that if you give the same, input, it reproduce the same error. So the correctness. Probability is zero, and I I think that there is a weak wing here. Ah! From from our colleague on that. But what a you know, if you want to elaborate On What do you mean about a narrow net correctness with some probabilities?
Michael Carbin: So yeah. So what is your event space? That's really the question right? Are we talking about? This? Probability is quantified over the distribution of inputs that may have passes in the system that that's one way to think about this, because otherwise yes, you're right. This is the deterministic at least a neural network as constructed is deterministic. There are certainly other techniques for randomizing behaviors that you're going to have coming out of a neural network. And so that's another alternative way of instantiating this probability
Michael Carbin: Um, personally. So I have played games with all sorts of different styles of these types of bounds. When we're doing the approximate computing work of can we, for example, again, quantify
Michael Carbin: What are we quantifying with That's always a fundamental question. Is it over outputs? We look at some of the work with hardware reliability, sometimes what it is is the stochasticity is going to be the hardware substrate itself. So it's subject to potential errors that we're going to have in the distribution over those.
Michael Carbin: But we've also even looked up techniques of randomizing the algorithms themselves. And so that way we're going to have a probability associated with that.
Dilma Da Silva: Um, We have now a question that relates to, you know, a different domain in terms of uncertainty. So possibly another extreme way. For example, with space shuttle control computers in those systems they would have a number. The the guest here said five independent computers, and they are configured to have this very high level redundancy, computing the same thing. You know the kind of thing that you you you mentioned as one
Michael Carbin: a way of doing that. And then there was another one computer that was completely independent computer and different group with different software, really meaning that it's not only replication, but there is
Michael Carbin: ah another entity that was trying to work with the same thing. So the question would be in, You know, when you see your research, what do you see yet to fit this domain, for example, flights computers
Dilma Da Silva: in computer control aviation. So if lives are in the line, or if it is really a space where reliability is higher than average softer, how do you see
Dilma Da Silva: space evolving regarding handling uncertainty,
Michael Carbin: and I hope I made a reasonable original presentation for me.
Michael Carbin: Yeah. So things like redundancy, i'm still core supportive of these, you know, some of the idea of doing approximation rather than redundancy is,
Michael Carbin: is illustrative of instances where perhaps, we can think differently, we don't necessarily always need our time to see if lives are online. Perhaps you don't want to be that it's not something you want to do. You don't accept this at all. Now, having said that there are other challenges thinking in
Michael Carbin: in this type of space. So there's a line of work that we've done, my student era excellent, and um has done what we called belief program, which the idea, and where we actually applied. This was with the Mars Polar liner and So the mars-polar liner is an interesting case study.
Michael Carbin: So their Gpl. Actually did a report out of this lander where the little Hendridge unfortunately Mars border Lander crashed into the surface.
Michael Carbin: Um! And so what happened?
Michael Carbin: And an abstract view is, the Lander was coming close to the surface.
Michael Carbin: It has a sensor system, a sensor and essentially detection system that's trying to detect whether or not it has landed on the surface. So, unfortunately, several hundred feet above the surface, it believed it had actually touched the surface, even though it had not.
Michael Carbin: And so it turned off at his engines, and therefore fell just into the into the surface.
Michael Carbin: Now, if you look at that report. What's interesting there is what happened was there? There was a failure to account for the uncertainty in the senses. So what happened?
Michael Carbin: Um! Is that the altimeter that they have is detecting the distance between the craft and the surface of the moon could actually be influenced by deploying the landing gears, Or, essentially what you do to these false positives that you had
Michael Carbin: now in designing a system. This is something they knew that was a problem, or, to say a a source of uncertainty and sensor design. They had written essentially an analysis of how the system should operate, what should be done, how to account for the uncertainty the fact that these false positives would happen. But in the software implementation they actually implemented it incorrectly.
Michael Carbin: And so this case wasn't handled appropriate
Michael Carbin: Now, in this line of work that we have. What we've done is we've come up with a way of a modeling language for someone to write down this sensor uncertainty, essentially saying, Okay, this sensor to produce this altitude. However,
Michael Carbin: if the if um if we deploy the landing gear, then it's possible that this sensor or this altitude or reading has been corrupted right? And so you can actually write this down essentially as a program.
Michael Carbin: And then what we do is we have a system that can take this uncertainty model
Michael Carbin: and actually dynamically track the insert? It can tell you. Is it possible, right, that there's an error with this sensor at this point in time, right? And if so, you can make a different decision. So what it does is actually automate some of the
Michael Carbin: or work that they were attempting to do there in the software implementation, but had failed to implement correctly. And so, when you think about uncertainty, at least at that safety critical level. I would say there are multiple sources. Yes, there's the reliability issue thinking about hardware reliability. So the natural liability of being in a sensor based environment. And in that environment,
Michael Carbin: in many instances there's a manual process for trying to manage that uncertainty that we think that there's a rise of new languages, things like published to programming. Like, I said, belief programming is what we had talked about, which,
Michael Carbin: as a similar concept of, instead of tracking probabilities of events, and states that you may be in It's going to track sets,
Michael Carbin: and then having systems that dynamically allow you to interrogate that uncertainty to then make decisions right without you needing to encode that sort of uncertainty yourself.
Dilma Da Silva: It's
Michael Carbin: that's That's quite interesting, and I think i'm going now to i'm not sure if i'm going to ask a question or give you a top of my, you know core
Dilma Da Silva: reaction to this whole story. So you know. So
Dilma Da Silva: people who have debugged from Gcc. Errors that only appears when you have a thousand cores or something like that.
Dilma Da Silva: It's been, you know, months, months, months trying to find this fast that of course, the designers didn't think about. So it was, you know, a corner case. So when i'm thinking now that you know my next project, I may decide to use a compiler from your group that is, you know, generated. And i'm thinking now on this one, how we can explore a behavior and try to do that, debugging that we did before, because we understood exactly like by line we could try to follow
Michael Carbin: the follow the interpretability by doing, tracing, as you mentioned in your talk. So that part of a problem determination becomes really hard. So when systems became really large, distributed systems explored, you would look at the top conferences in systems, Osdi sosb, And there are so many papers on helping people to find problems on those systems.
Michael Carbin: They became it right really hard. Now, when I look that if we do go, you know, Surrogate,
Dilma Da Silva: it's part of the methodology. I'm not even saying that I heard you that it were replaced. But let's say that now the software developer developed me a new file system. Then I will have a piece that is a surrogate to capture some things, and it will have other things that may come from, you know Posix Semantics, or whatever you know, legacy you need, and i'm going to marry. The hard part is that what is this story for problem determination? Who are the people who get the call? And now there is this problem, and it's really high, and we need a swath team to fix and find it
Michael Carbin: how we're going to work on it. And I wonder if, when you're thinking about the program with the dollars you're thinking about that about as you put pieces together. If there is some breadcrumbs or some traces that you can give to help people to to find the problems, because that may be really hard. Now, another part of it
Dilma Da Silva: is that. Uh,
Dilma Da Silva: if you look at things like file systems. Most files are small and die soon, right? So a lot of our file systems are inefficient. If you go to the worst possible case scenario to guarantee correctness or file systems. But in your very, you know, approach. I was trying to just to find the distribution. They put data that you said, That is a hard problem how you do those Ah, ah data sets in order to get to some notion of reliability how that all fits together, since we may not
Michael Carbin: see those Corner Corner Corner cases very common. So the reliability i'm repeating your last slide, just saying that I think this reliability and this this story would be a big challenge mine. So you probably have more answers about, You know you probably don't see as many
Dilma Da Silva: her obstacles as I see right now.
Michael Carbin: Uh, no, I I see lots of obstacles, right, I,
Michael Carbin: as you pointed out on the dip-tune side you know this. This was part of our inspiration. How can we make this interpret? How can we still
Michael Carbin: use the power of data, but give a description that essentially fully interpretable? So the output of dip tune is just a tuned version of Ah Lvm Mmca. Where you can interrogate. Still, essentially, all the decisions that are being made. So the final output is not something where you need to understand what? An our our system
Michael Carbin: right now more broadly than that
Michael Carbin: it is.
Michael Carbin: It's going to be a fundamentally interesting future. I have to say I I
Michael Carbin: I think the forces of software developing are moving in a direction where
Michael Carbin: to access new capabilities. These types of
Michael Carbin: perhaps naturally uninterpretable components are going to be integrated. Right?
Michael Carbin: So
Michael Carbin: I think the challenges you're outlining are ones that we have to face and be prepared to face right? And think of ways of enabling people to lose this type of technology in ways that overall give a good understanding of the system, right? So it's. I
Michael Carbin: I think these are all I don't know if i'm hopeful. I think it's unavoidable
Michael Carbin: Right? I think the power of these tools put in people's hands are such that they can do things that they couldn't do before, and so they will do that.
Michael Carbin: And so that that's where we're going to end up. And so what comment are we going to make? We can tell people not to do it. But I I don't think that's really going to help us
Dilma Da Silva: in the in your talk. Ah, that's a ah question here. Ah, there was an example of a program that was mixing cold and English, you know, saying that this is a way of doing ah becoming a common way of developing software. And you, I think you showed an example. But it went ah fast to read. So you want to tell it to tell us more about an example of a programs people are holding now where they have really just natural language as piece of it.
Michael Carbin: So there there was an example that I was seeing where someone was trying to write.
Michael Carbin: Essentially, After that there are a few things. So
Michael Carbin: one example is actually a student mine who was working on a a search essentially a a semantic search, a semantic, abstract search of of art.
Michael Carbin: So he's got this program. That's largely okay. There are a couple of keywords that I want to put together, and here is suddenly this pl: There's a prompt out to this large language model that says that I want you to, you know, search for a whole bunch of papers that, let's say, have cinema of all these words that I have here, and then bring those back to
Michael Carbin: so cinema's of this word,
Michael Carbin: where you know word is something that's been written. This is something that's not coming as input, but synonyms of is an algorithm that is now being interpreted by a large language model.
Michael Carbin: I. There's we we just short of encoding all possible synonyms and
Michael Carbin: right of the words that we have and expected's inputs. This is not something that's going to be programmed manually, but can be written in English, given an intuitive understanding, and can be interpreted by a large language model. To then go and retrieve results that are related to that.
Michael Carbin: So these are the types of computations, I think, are
Michael Carbin: in the short term being integrated where we can give natural language specifications to what they do. But the programming implementation is I mean it's. I guess you can go and try to develop a massive thesaurus and do this. Or perhaps there's an available source that's out there. But this can just be done automatically.
Michael Carbin: It's
Dilma Da Silva: in that. That's okay. That's reasonable.
Dilma Da Silva: Okay? I think we Ah, we don't have Ah, more questions. Ah, for the Nsf. People! We'll be talking to Mike again at three Pm. Eastern. Ah, for a little bit, If we have, you know people to discuss more. Ah, but thank you so much. Thank you for people who attended.