WEBVTT 00:00:07.000 --> 00:00:16.000 3, dash 5, 62. We're gonna do a brief description in the beginning, and then we're gonna answer questions through the chat. 00:00:16.000 --> 00:00:16.000 So if you have questions that you're thinking about, please feel free to start typing in the chat. 00:00:16.000 --> 00:00:25.000 Now hopefully, we'll answer those questions as we go. 00:00:25.000 --> 00:00:35.000 But please start typing your question. So I'm gonna introduce my team here and actually, Jay, I'll start. Jay, would you like to introduce yourself? 00:00:35.000 --> 00:00:45.000 The the program offices in the is division. The passing type is cluster. 00:00:45.000 --> 00:00:47.000 Okay. And then India. 00:00:47.000 --> 00:00:55.000 Hello! I'm on in the battery. I'm in size in the computing and computer foundation division. 00:00:55.000 --> 00:00:59.000 And I'm in the software and hardware foundations cluster. 00:00:59.000 --> 00:01:02.000 And David Gorman. 00:01:02.000 --> 00:01:08.000 Yeah, thanks. I'm David Corman I'm part of the Cns division in size. 00:01:08.000 --> 00:01:14.000 And I also lead our cyber fiscal system program. 00:01:14.000 --> 00:01:17.000 I don't believe Pavitra is here today. 00:01:17.000 --> 00:01:27.000 I think she's she was double booked, so the visa probably car is one of our program officers, and our communication and computing and communication foundation. 00:01:27.000 --> 00:01:33.000 So an important part of our team. 00:01:33.000 --> 00:01:45.000 Oops. So we we're oops sorry. Sorry, so we're all the important thing in this slide is really to realize that all of size is taking part in this. 00:01:45.000 --> 00:01:43.000 So we see that there's really important aspects of each of the divisions. 00:01:43.000 --> 00:02:02.000 So whether it's communicating and communication foundations, whether it's computer network systems or whether it's information and intelligence systems, we see aspects of all of these areas in here. 00:02:02.000 --> 00:02:09.000 So that's why this is a cross. This whole initiative is across size, because it's important in so many different ways. 00:02:09.000 --> 00:02:09.000 I also wanna introduce our partners or have our partners introduce themselves? 00:02:09.000 --> 00:02:23.000 So from open fillet philanthropy projects. So Asia Cootra is one of our colleagues. 00:02:23.000 --> 00:02:31.000 Hi! I'm Ajaya! I am a Grant maker at the open Philanthropy project, focusing on AI safety issues. 00:02:31.000 --> 00:02:37.000 And our my partner, Dan Hendrix, is gonna be doing most of the talking. 00:02:37.000 --> 00:02:43.000 He has a lot of experience in in the AI safety field, and he runs the center for as safety. 00:02:43.000 --> 00:02:46.000 Yeah. So I'm collaborating with open philanthropy on this. 00:02:46.000 --> 00:02:58.000 And I really excited to see so many people interested in safety, especially now, as the broader world is introduced in it too, and the public is wanting some safety research. 00:02:58.000 --> 00:03:02.000 So I'm glad everybody's here, and, thanks to the Nsf. 00:03:02.000 --> 00:03:04.000 For doing this initiative. 00:03:04.000 --> 00:03:07.000 Yeah, thank you so much. It's very exciting. 00:03:07.000 --> 00:03:11.000 And thanks to our partners with our partners, we don't get to do the things we wanna do. 00:03:11.000 --> 00:03:12.000 And the things that we all think are important. So we're really very grateful to our partners on this. 00:03:12.000 --> 00:03:17.000 So thank you. So I think, in Jan just alluded to, our AI systems are increasing. 00:03:17.000 --> 00:03:22.000 Rapidly, and we're seeing all sorts of new things happening. 00:03:22.000 --> 00:03:35.000 But there, we know they're deployed at high stakes, and we see this safety issue is becoming extremely important, as if it wasn't before. 00:03:35.000 --> 00:03:41.000 But I think as we see the proliferation of AI, we really this safety issues become extreme. 00:03:41.000 --> 00:03:48.000 And we really wanna do more you know, we we wanna make sure accuracy. 00:03:48.000 --> 00:03:53.000 Efficiency is scalability are done. But we really wanna make sure that they are. 00:03:53.000 --> 00:04:16.000 These systems are robust. So when we think about extreme events, we think about monitoring for stranger and safe behavior, we really wanna start designing systems that are built with this in mind that are really that are coming prepared to think about safety as one of the first features, so we thought in a safety, we should have we should 00:04:16.000 --> 00:04:29.000 have a joke, but I from one. It's not the happiest to judge, but I think the thing we that we included it on was, but we really have to think about safety is not happening by chance safety is it's got to be a critical. 00:04:29.000 --> 00:04:35.000 Component. So so given all this, we really wanna think about what are the undecirable system behaviors that are safety issues. 00:04:35.000 --> 00:04:56.000 So what kind of things did they? They have? So we think about over blunders, prediction, error, system crashes, silent failures, like reporting on justified confidence levels out of distribution issues and competently achieving unintended objectives. 00:04:56.000 --> 00:05:16.000 So we want people to start to think about these undesirable behaviors and make sure, as we deploy systems, that these undesirable behaviors are coming to pass and that they're identified before that cause a safety issue so in our learning enabled systems we have learning components that include 00:05:16.000 --> 00:05:22.000 but remember, these are not limited to deployed system, and in health care, medicine. 00:05:22.000 --> 00:05:29.000 We can have them in criminal justice. We can have a monomonomous and cyber physical system and finance. 00:05:29.000 --> 00:05:36.000 Remember, as long as these in health care medicine we can have them in criminal justice, we can have them autonomous and cyber, physical systems and finance. 00:05:36.000 --> 00:05:44.000 Remember, as long as these learning enabled systems are, are a safety issue, they can be in any one of the Dom but these also include foundational learning based systems that may be subsequently applied to many downstream domains. 00:05:44.000 --> 00:05:52.000 So the topic we're talking about is broad. But again we remember that we're really thinking about the safety component. 00:05:52.000 --> 00:06:07.000 So what we're trying to do really here, and working with open fill and Nsf is soliciting foundational research that that leads to learning and enable systems with which safety is really done with a high level confidence. 00:06:07.000 --> 00:06:12.000 Now, I know that seems like it's obvious, but I think we've all seen that. 00:06:12.000 --> 00:06:15.000 That's not the case as we're doing deployment. 00:06:15.000 --> 00:06:17.000 So this is what we're trying to get with this. 00:06:17.000 --> 00:06:23.000 We're trying to build the science base that gets us these high levels of confidence in these learning systems. 00:06:23.000 --> 00:06:23.000 So we'll consider it as the success. This program really works. 00:06:23.000 --> 00:06:41.000 If developers are the future learning and enabled systems, can one informally explain why the system can be deployed safely and then unpredictable environment, and then back these informal explanations with rigorous evidence that the system satisfies safety specifications? 00:06:41.000 --> 00:06:54.000 So you can tell the general public why it can be deployed safely, and you can back it up with the data and the evidence for that. 00:06:54.000 --> 00:06:54.000 So the program is looking for proposals that advance. You can do it. 00:06:54.000 --> 00:07:04.000 This general theories, principles, and methodologies to get to these safe learning enabled systems. 00:07:04.000 --> 00:07:08.000 And we want to get beyond the single problem. Specific problem. Instance. 00:07:08.000 --> 00:07:13.000 So we wanted, we really want to go deeper than just one simple, one simple problem. 00:07:13.000 --> 00:07:17.000 We want to really think about safety is this whole issue, and we've gone to think about it. 00:07:17.000 --> 00:07:23.000 It's as we do scalability and deployability. 00:07:23.000 --> 00:07:38.000 So ideals for the program really are that proposals have the potential to make strong advances in the design and implementation of safe learning enabled systems as well as re-advancing methods for reasoning about the safety. 00:07:38.000 --> 00:07:37.000 So we know we need new methods. We know we need ways to think about this. 00:07:37.000 --> 00:07:48.000 And we're hoping that this initiative really brings people with new ideas to the table. 00:07:48.000 --> 00:08:06.000 So ideal, will demonstrate how these 2 objectives will be achieved, provide evidence that the proposed approach will improve notions of safety, and then argue that potential for lasting interest on both rigorous safety evaluation methods and on the design and implementation of safe learning 00:08:06.000 --> 00:08:09.000 enabled systems. 00:08:09.000 --> 00:08:15.000 So our safety guarantees here, we wanted one of the things we wanna verify that the learning system achieve safety guarantees for all possible we know that that's gonna be difficult. 00:08:15.000 --> 00:08:25.000 But we really that's, you know. Obviously, that's our goal. 00:08:25.000 --> 00:08:43.000 But we know that's not gonna happen. So we really wanna think about consider considerations for establishing safety guarantees, systematic generation of data from realistic but yet appropriately pessimistic operating environments, resilience to unnat unknowns so monitoring for 00:08:43.000 --> 00:08:58.000 hazards or behaviors, new measured methods for reverse engineering, inspecting and interpreting the internal logic of these learn models, and then method for improving the performance of, or by directly adapting the system's internal logic so these are what we're looking 00:08:58.000 --> 00:09:03.000 for. And we're thinking about establishing safety barriers. 00:09:03.000 --> 00:09:16.000 So our safety requirements, any system claiming to satisfy a safety specification must provide rigorous evidence through analysis corroborated empirically or with a mathematical proof. 00:09:16.000 --> 00:09:15.000 So we can do this empirically or with a mathematical proof. 00:09:15.000 --> 00:09:19.000 So we can do this empirically or with a mathematical proof. 00:09:19.000 --> 00:09:25.000 But we have to provide rigorous evidence. That's important to this initiative. 00:09:25.000 --> 00:09:31.000 Proposals also, and that increase safety primarily as a downstream effect of improving standard systems. 00:09:31.000 --> 00:09:37.000 Performance, metrics unrelated to safety, such as accuracy on a standard task or not in scope. 00:09:37.000 --> 00:09:39.000 We have planned of initiatives. We're improving system. 00:09:39.000 --> 00:09:51.000 Performance is part of what we do in this. We're really looking for ones that directly impact safety. 00:09:51.000 --> 00:09:58.000 So if you looked in our solicitation, you would see that all proposals have 4 things you have to talk about. 00:09:58.000 --> 00:09:59.000 The notion, of end to end mathematically or empirically based safety to it. 00:09:59.000 --> 00:10:03.000 In plain English, what do you? What do you? What are you doing? 00:10:03.000 --> 00:10:14.000 How are you talking about this? And then justify why the end-to-end safety product properties are critical in this system. 00:10:14.000 --> 00:10:15.000 And that's really important. We don't want downstream effects of just improving performance. 00:10:15.000 --> 00:10:29.000 Really? What about the end? Safety properties on this identify the environmental assumption for the safety property. So in what conditions, what? 00:10:29.000 --> 00:10:33.000 What are, where, what is the issues that we're facing? 00:10:33.000 --> 00:10:42.000 And what are the automated, semi automated or interactive techniques for establishing the degree to which the safety project properties are satisfied? 00:10:42.000 --> 00:11:00.000 Again, we know it's hard to satisfy all the possible options, but we really want you to come to us saying what you think you're doing there, which which degree the safety properties are satisfied, and then demonstrate these techniques techniques achieve safety and again. 00:11:00.000 --> 00:11:09.000 it's mathematically or empirically through rigorous simulation, prototyping integration with actual learning enabled systems. 00:11:09.000 --> 00:11:18.000 So again, when I we talk about notions of safety, we're talking about robustness and resilience the tail risk monitoring systems for anomalous. 00:11:18.000 --> 00:11:29.000 And unsafe behavior, interpreting reverse engineering, and expecting a learn systems, internal logic or reliability under human error. 00:11:29.000 --> 00:11:32.000 So! 00:11:32.000 --> 00:11:38.000 So when we do the Nsf Review criteria, of course we're talking intellectual merit and broader impacts. 00:11:38.000 --> 00:11:42.000 And those are the same criteria from the National Science Board. 00:11:42.000 --> 00:11:55.000 So? The first question we ask of all of our panelists is, what's the potential for the proposed activity to advance knowledge and understanding what within its own field or across different fields? 00:11:55.000 --> 00:12:00.000 We also for broader impacts, talk about the benefit to society or advanced societal outcomes to advance. 00:12:00.000 --> 00:12:08.000 What happens? We talk about? What are the new creative or potentially transformative concepts? 00:12:08.000 --> 00:12:12.000 Is the plan well, reason that sort of sound rationale for this. 00:12:12.000 --> 00:12:12.000 How qualified is the individual team or organization? And are there adequate resources? 00:12:12.000 --> 00:12:24.000 So those are a standard review curriculum. 00:12:24.000 --> 00:12:30.000 For this one. We also say that the proposal will also be evaluated, based on the components. 00:12:30.000 --> 00:12:33.000 So we want you to discuss learning enabled components. 00:12:33.000 --> 00:12:40.000 The proposal should describe the components and provide research why they are appropriate for the system being stuck. 00:12:40.000 --> 00:12:40.000 Sometimes you can't study the whole system, but we want you to think through. 00:12:40.000 --> 00:12:44.000 What components you're studying, and why, and make the case. 00:12:44.000 --> 00:12:51.000 Why, that's the appropriate it's it's components to test. 00:12:51.000 --> 00:13:01.000 We also again come back to the rationale and clean language of why do the end and safety properties are critical to this in this system? 00:13:01.000 --> 00:13:09.000 We also want the safety plan. Again, those in the environmental assumptions under which these safety properties are ensured. 00:13:09.000 --> 00:13:09.000 So you know, we're thinking about what are the techniques there? 00:13:09.000 --> 00:13:16.000 And what are the what are you doing? So that's our safety plan. 00:13:16.000 --> 00:13:19.000 And then validation, so you need a plan to validate these techniques, and that might be mathematically or empirically. 00:13:19.000 --> 00:13:30.000 It might be through rigorous simulation prototyping, and integration, with actual including sub scale learning enabled systems. 00:13:30.000 --> 00:13:39.000 What is really important here is for the synergy projects, the validation plan must include experimentation on actual learning enabled systems. 00:13:39.000 --> 00:13:45.000 So for the synergy only that's it must include experimentation. 00:13:45.000 --> 00:13:51.000 So with that we can move to questions. We have a list serve. 00:13:51.000 --> 00:14:02.000 If you need the listserv. So with that, I'm gonna open this up for questions, and we have a great team here to answer your questions. 00:14:02.000 --> 00:14:11.000 So alright, so! 00:14:11.000 --> 00:14:17.000 Okay. So in the. 00:14:17.000 --> 00:14:34.000 So there's a question here, it says, is there a criterion for us to decide whether one specific application qualifies for an actual learning enabled system? 00:14:34.000 --> 00:14:37.000 Anybody. 00:14:37.000 --> 00:14:48.000 So I can. I can answer that for part of the give at least part of the question, especially when we think about the synergy element. 00:14:48.000 --> 00:15:05.000 So ideally, the ideal case is, you have your research, you create an experimentation plan, and you evaluate what you've defined as safety properties on a real learning enabled system. 00:15:05.000 --> 00:15:26.000 A. For example, autonomous vehicle, sure, short of owning an experimentation platform like a large autonomous vehicle, a sub scale, autonomous vehicle that's still includes the basic you can run your experiments. 00:15:26.000 --> 00:15:49.000 And that qualifies, or for that, it's a system that has the sensing has the processing, and you can create experiments where you challenge your safety criteria. 00:15:49.000 --> 00:15:55.000 Thank you, David. Anybody else want to comment on that one. 00:15:55.000 --> 00:16:09.000 Alright. There's another question here about, does this program allow for research on existing systems, such as Chat Gpt versus developing new systems? 00:16:09.000 --> 00:16:14.000 So do you? Wanna if somebody wanna answer that part of the question. 00:16:14.000 --> 00:16:25.000 I mean, I think large language models are quite relevant for existing applications, and you could come up with some approved model or some modified model using using some existing large language models. 00:16:25.000 --> 00:16:22.000 Make those safer. So we need to create a new one from Scratch. 00:16:22.000 --> 00:16:35.000 There are many ways that we could understand large-language models better. 00:16:35.000 --> 00:16:43.000 We could try and understand their internal logic by finding mechanisms inside of them that explain how they're doing some of their decision making to give us more confidence whether they're reasoning is appropriate. 00:16:43.000 --> 00:16:48.000 There are other things you could do on top of Chat. 00:16:48.000 --> 00:16:56.000 Gbt like. See if it's if it's encountering an anomalous scenario, or whether we should trust it in a different type of condition. 00:16:56.000 --> 00:17:02.000 Those are all things you could use to study models like Chat Gp. 00:17:02.000 --> 00:17:06.000 But you could easily place, replace it with some other large language model. 00:17:06.000 --> 00:17:16.000 I think if we're concerned about real world applications, then the large language models are an interesting new one that are basically touching on almost every field. 00:17:16.000 --> 00:17:21.000 So that seems to have be a particularly interesting type of model to study. 00:17:21.000 --> 00:17:27.000 If we're trying to make a system safer in the real world. 00:17:27.000 --> 00:17:43.000 Yeah, I would add to that that the text of the the call says that foundation models on which further downstream applications could be built are in scope, and I think large language models like Chat Gpt and others would count for that. 00:17:43.000 --> 00:17:53.000 Yeah, I went to I, the purpose of this program is not divide by any specific system or applications, but the focus on safety issues. 00:17:53.000 --> 00:18:02.000 So the problem can come from the existing system, or technology, or the you approach it to those other applications. 00:18:02.000 --> 00:18:10.000 Anything you can define as a safety program that could. Where do I choose this program? 00:18:10.000 --> 00:18:14.000 So I think that leads into the next question, how broadly is safety define safety? 00:18:14.000 --> 00:18:24.000 Consider things like limitations to critical thinking or other more psychological harms versus harms to data, privacy, etc. 00:18:24.000 --> 00:18:27.000 Well, so we include in it a discussion of uncertainty. 00:18:27.000 --> 00:18:36.000 So if we can have models, have calibrated predictions, and if they're if their uncertainty is interpretable that can help people make better decisions. 00:18:36.000 --> 00:18:57.000 So if we know when to trust their model, and when it's more likely to be mistaken, and if we can get that to be highly reliable, and that would be useful likewise, if we can have some sort some guarantees on its behavior like certifies robustness certifications that can 00:18:57.000 --> 00:19:19.000 also, help. People know when to use it, and that it also touches on another question about how these would integrate into socio-technical systems by creating some of these safety measures and knowing knowing when to trust the model that can help a human operators more wisely to decide when to 00:19:19.000 --> 00:19:41.000 trust the systems, and I suppose, likewise with, if we can provide explanations for how they're arriving at their decisions and are able to interpret interpret the internal processes inside of models that can also help people know when to trust these models and address social technical issues. 00:19:41.000 --> 00:19:40.000 Hey! David! 00:19:40.000 --> 00:19:48.000 So we can. Can I add a slightly other additional dimensions for that question? 00:19:48.000 --> 00:20:01.000 There's line. We put language in there that says you should be able to express the safety properties in terms of playing English, and what we in part want. 00:20:01.000 --> 00:20:08.000 There is you the panel that's reviewing as well as the program. 00:20:08.000 --> 00:20:13.000 Directors should not be thinking of this as the riddle of the space. 00:20:13.000 --> 00:20:24.000 It should be very clear to us from your description. What are the saving properties, and why they are safety properties. 00:20:24.000 --> 00:20:30.000 It shouldn't be a question that we struggle with understanding. 00:20:30.000 --> 00:20:38.000 And I suppose building on that one thing that if somebody were to say, well, we'll reduce the error rate, or something like that, and that's the safety proposal. 00:20:38.000 --> 00:20:38.000 Well, that's a bit that's potentially too broad. 00:20:38.000 --> 00:20:46.000 Error rates about. What is this particularly related to any harm or hazards? 00:20:46.000 --> 00:20:52.000 Is this generically increasing the image, that accuracy or the knowledge of language models as they're answering. 00:20:52.000 --> 00:21:02.000 You know, exam questions, that sort of stuff is it distinctly about safety? 00:21:02.000 --> 00:21:07.000 And there are many other sort of programs for that. So I should be. 00:21:07.000 --> 00:21:15.000 So I agree should be, and the text confirms that it should be clear that it's distinctly focusing on safety and not making models. 00:21:15.000 --> 00:21:22.000 Doing doing this sort of research that people have been doing for the past several years, and just more of the same. 00:21:22.000 --> 00:21:30.000 So I think this the question is, it's not only physical safety, but you have to define your safety criteria, and you have to make it clear. 00:21:30.000 --> 00:21:36.000 So it has to be able to be understood and it can't just be improving performance unless you're dying. 00:21:37.000 --> 00:21:40.000 That you have your time, that to safety. Is that my summarizing this? Well? 00:21:40.000 --> 00:21:44.000 Yeah, also, I want to add there some issues. 00:21:44.000 --> 00:21:49.000 A lot include this program like a security fairness. 00:21:49.000 --> 00:21:53.000 I think so. Those are probably related to safety. 00:21:53.000 --> 00:21:56.000 But it's a lot I mean, let's not include in this program. 00:21:56.000 --> 00:22:00.000 We have made a career in the in the summer station. 00:22:00.000 --> 00:22:11.000 On security. We do have some language about appropriately pessimistic environments, though, and simultaneously, in doing simulations or trying to come with hard examples that they'd be appropriately rigorous. 00:22:11.000 --> 00:22:27.000 So you still still, there's certainly a hope for doing some rigorous stress tests, testing of the system. So. 00:22:27.000 --> 00:22:52.000 Sorry I shouldn't mute my cell phone. There was a question about what's the criteria to decide whether one specific application qualifies for an actual learning enabled system. 00:22:52.000 --> 00:22:58.000 I think the question is, is there are there we're talking about. 00:22:58.000 --> 00:23:01.000 We're we should learn more than a single application. 00:23:01.000 --> 00:23:11.000 But they but the learning system may be one system, right? We can have pieces of a larger system, or we can have a full learning system. 00:23:11.000 --> 00:23:19.000 So you can imagine this in many different ways. Do you guys wanna give some example? And ninja? 00:23:19.000 --> 00:23:30.000 I was just going to say, as the civilization says, a learning enabled system is one that has, and that is machine learning components. 00:23:30.000 --> 00:23:35.000 So that can be, you know, it can be very broadly interpreted. 00:23:35.000 --> 00:23:47.000 So whatever it, the system with machine learning components is going to be considered. Learning enabled system. 00:23:47.000 --> 00:23:55.000 So chat, gpt, or ama would consequently yeah fit that bill. 00:23:55.000 --> 00:23:59.000 Alright! 00:23:59.000 --> 00:24:03.000 So there's a question here. It says, components on the component issue. 00:24:03.000 --> 00:24:09.000 The proposal, describe the system components and provide reasons why they are appropriate for the system being studied. 00:24:09.000 --> 00:24:20.000 What does the system mean? Here? The deployable AI system? 00:24:20.000 --> 00:24:29.000 I think it could possibly be interpreted into it, so it could be that they are system, or you could imagine the larger system where there are some human operators involved. 00:24:29.000 --> 00:24:48.000 So if we're if the model, if the learning enabled model is itself outputting things like confidence measures or an indication that there's anomalous behavior going on, or that it there's potential like, for instance, maybe a model is gaming and objective or pursuing an 00:24:48.000 --> 00:25:04.000 unintended objective. Maybe if there are features that enable people to catch those issues and reduce exposure to those hazards that that could also improve safety. 00:25:04.000 --> 00:25:02.000 Go ahead! 00:25:02.000 --> 00:25:06.000 I'd add to that. Oh, oh, I'd add to that! 00:25:06.000 --> 00:25:13.000 But one type of system, that open plan to be has particular interest in is that these large language models are starting to assist in coding workflow. 00:25:13.000 --> 00:25:14.000 So they're starting to write code at the request of humans which humans then run or do other things with. 00:25:14.000 --> 00:25:24.000 So I think it would, thinking about how the outputs are used. 00:25:24.000 --> 00:25:27.000 Are they just gonna be run? Are they gonna be monitored in some way? 00:25:27.000 --> 00:25:36.000 Could be construed as like the system that this machine learning model is part of. 00:25:36.000 --> 00:25:39.000 Yeah, as humans are being more and more removed from the loop. 00:25:39.000 --> 00:25:53.000 And as people are using AI systems to replicate a lot of that functionality, the notion of what the AI system is, seems to be expanding, and it seems quite important to keep track of how it could be creating new new sorts of problems for Us. 00:25:53.000 --> 00:26:05.000 As as it replaces a much functionality. 00:26:05.000 --> 00:26:14.000 The next question is the notion of fairness, not being able to provide recourse considered as safety. 00:26:14.000 --> 00:26:22.000 So there's the notion of safety more broadly, but then there's what was in the in this solicitation. 00:26:22.000 --> 00:26:28.000 So of course, there's the problem of alignment to human values, things like well-being and impartiality, people getting what they deserve and what not. 00:26:28.000 --> 00:26:42.000 But in this solicitation there's generally more restricted to deal with to deal with more standard harms and hazards. 00:26:42.000 --> 00:26:49.000 There are generally other proposals and programs that get an issue such as fairness, though. 00:26:49.000 --> 00:26:50.000 So this isn't to say that this is some like really strong intellectual stance. 00:26:50.000 --> 00:27:05.000 Against against other sorts of human values primarily, that there's only so much the program can fund. 00:27:05.000 --> 00:27:22.000 So I. The next question is, when learning enabled systems are embedded in socio-technical systems, so just some of the examples in the safety is arguably not just about the behavior and the technical system, but how it's used by people how much of this is in scope, or is this intended to be 00:27:22.000 --> 00:27:26.000 purely technical. 00:27:26.000 --> 00:27:38.000 Well, it's partly mentioned before. There are plenty of technical things that can be to help with the socio-technical aspect of people not unduly trusting these systems. 00:27:38.000 --> 00:27:56.000 If we have a better idea of their internal workings, or when they're likely to be useful or when they're likely to succeed or helping human operators monitor these systems more effectively, all those help with very various socio-technical aspects so there are plenty of there are 00:27:56.000 --> 00:28:11.000 plenty of technical safety measures one can do to improve broader systemic factors that affect whether there'll be accidents or it could be caused by these systems. 00:28:11.000 --> 00:28:14.000 I I think that we're really looking in this one. 00:28:14.000 --> 00:28:19.000 It's more of the technical than it is of the societal. 00:28:19.000 --> 00:28:21.000 So I think that. 00:28:21.000 --> 00:28:26.000 Yeah, I as a that, basically, we have, you have to be clear. 00:28:26.000 --> 00:28:30.000 Define the problem mathematically, or experiment to me right? 00:28:30.000 --> 00:28:36.000 So you needed to create, defined. So technically, sometimes they are easier to define. 00:28:36.000 --> 00:28:38.000 Of course, if you can come up with a definition for the first, for other, we are open to any safety problems. 00:28:38.000 --> 00:28:47.000 As far as you can define community, and you have the way to verify it. 00:28:47.000 --> 00:28:49.000 That's what you find. 00:28:49.000 --> 00:28:49.000 So, yeah, the solicitation. Okay? 00:28:49.000 --> 00:28:53.000 Yeah. Go ahead. 00:28:53.000 --> 00:29:12.000 Oh, thank you. The solicitation says that you need to define the particular safety property you're going for in plain English, and that property can make references to how the system is used, so you can say, even if a user uses the system in in X reckless way the system should not do why 00:29:12.000 --> 00:29:12.000 bad thing that that can kind of make reference to the socio-technical aspect. 00:29:12.000 --> 00:29:26.000 But then the particular solutions, like Dan said, are are, and Wendy said, are more of the like technical solutions to address. 00:29:26.000 --> 00:29:25.000 Yeah. 00:29:25.000 --> 00:29:30.000 And and as well in the word, in the wording. There's end to end system, too. 00:29:30.000 --> 00:29:39.000 So we're, of course, trying to think about how it touches on the touches on the system. In context. 00:29:39.000 --> 00:29:39.000 Oh, if I can add, one has to be very clear about what what are safe behavior? 00:29:39.000 --> 00:29:52.000 And the system, and then 1 one way of thinking about that is is just, you know. 00:29:52.000 --> 00:29:56.000 If if you think about the semantics of the system, what are the traces? 00:29:56.000 --> 00:30:01.000 What do the traces of that system look like? What are the safe traces? 00:30:01.000 --> 00:30:12.000 What are the unsafe trade? What is? Yeah? So these are issues that on may consider. 00:30:12.000 --> 00:30:18.000 Okay, we've got a lot of questions about the specific definition of safety. 00:30:18.000 --> 00:30:23.000 And the question is, is it just physical harm? 00:30:23.000 --> 00:30:27.000 And I think we've gone through that. But. 00:30:27.000 --> 00:30:31.000 Do we want to add anymore the definition of safety? 00:30:31.000 --> 00:30:32.000 So safety isn't just making the system do perform better. 00:30:32.000 --> 00:30:46.000 There are many ways in which you could have systems, have fewer errors, and that would in some ways make this system safer, but then they also might be more capable at doing things that are potentially hazardous. 00:30:46.000 --> 00:31:08.000 So, that that's why there's a specific, a text in the in the solicitation stating that if you're primarily just improving safety as a downstream, effect of improving the general performance of a system it's generally a rate, and things like that that sort, of thing is not in 00:31:08.000 --> 00:31:12.000 scope so safety isn't just making the system work better. 00:31:12.000 --> 00:31:18.000 You need to show how you're specifically focusing on some targeting safety and not targeting. 00:31:18.000 --> 00:31:25.000 It's general overall, diffuse performance. 00:31:25.000 --> 00:31:28.000 Anybody want to add to that? 00:31:28.000 --> 00:31:47.000 Just simply, you know, agreeing with Dan. Yeah, really, think about the plain English element we're asking you, you know, the proposer to tell us what is safety in your in your mind. 00:31:47.000 --> 00:31:56.000 What are you, considering? Safety and provide a wow. Reason easy to understand. 00:31:56.000 --> 00:32:11.000 Discussion of why that is safety, and then be able to show, either through imperical or mathematical, that your system enables, proving that. 00:32:11.000 --> 00:32:17.000 Yes, I'm meeting these criteria. It is a safe system. 00:32:17.000 --> 00:32:17.000 Hey! 00:32:17.000 --> 00:32:25.000 And you know, in part that's the end-to-end part that Dan touched on is really, you may. 00:32:25.000 --> 00:32:35.000 Only what we don't want is you to consider only a single component of an integrated system. 00:32:35.000 --> 00:32:40.000 You may be other reason here is that behavior of this component. 00:32:40.000 --> 00:32:43.000 But how does that component, then, influence the overall system? 00:32:43.000 --> 00:32:50.000 Safety! 00:32:50.000 --> 00:33:03.000 Thanks. Thank you all for that. One of the questions here is so an actual learning enabled system doesn't have to already exist in the world, but can emerge from the project. 00:33:03.000 --> 00:33:03.000 I think we're not looking for a new system. We're looking for a safety. 00:33:03.000 --> 00:33:12.000 These are really targeting safety. So if you're gonna create a new system, and then you're gonna have to build it. 00:33:12.000 --> 00:33:22.000 So it's not safe. It's gonna take a lot of twisting and turning in your proposal to get there. 00:33:22.000 --> 00:33:22.000 So you might wanna pick a learning enabled system. 00:33:22.000 --> 00:33:25.000 And then you can address the safety and and spend the proposal and the funds working on the safety issues. 00:33:25.000 --> 00:33:37.000 Not building a new system. So I'm sure there's a possibility. 00:33:37.000 --> 00:33:42.000 But I think it seems like a lot of extra work for not getting to safety. 00:33:42.000 --> 00:33:50.000 So so is AI alignment considered in scope for the solicitation. 00:33:50.000 --> 00:33:56.000 Well, there's there's some text like studying whether models are competent to achieving unintended objectives. 00:33:56.000 --> 00:34:15.000 So what we could imagine, say, using a large language model. That's pursuing some that's taking some actions, and maybe some text based environment and maybe wanting to study some of its safety properties of how it's interacting within its environment and is it causing harms in that environment that might 00:34:15.000 --> 00:34:27.000 be an interesting that might be an interesting microcosm that would touch on, that would touch on alignment issues in some capacity. 00:34:27.000 --> 00:34:33.000 Okay, so we do have a question about, is it? 00:34:33.000 --> 00:34:49.000 So there's a question about, are we we looking at safety at the level of systems, or are are we looking at individual system, misbehavior, or at a broader scale than that? 00:34:49.000 --> 00:34:55.000 It's hard to translate this question. So. 00:34:55.000 --> 00:35:13.000 The kind of the wholesale harms of these learning systems to society, and I will start with, as we're asking you to describe your notion of safety you're gonna try to do all of society it's gonna be hard to be able to verify that to test 00:35:13.000 --> 00:35:21.000 that what we're looking for is, generalize little information that we get do in these learning enabled systems. 00:35:21.000 --> 00:35:24.000 So we want you to come up with new methods, new ways to look at. 00:35:24.000 --> 00:35:30.000 This new ways to understand it. Then we could be able to think about across systems. 00:35:30.000 --> 00:35:39.000 You all. Wanna add to that? 00:35:39.000 --> 00:35:44.000 Alright. Could you all elaborate on the high stakes, scenarios? 00:35:44.000 --> 00:35:50.000 The program has in mind. 00:35:50.000 --> 00:35:50.000 Well, there. There are many high stakes scenarios right now. 00:35:50.000 --> 00:36:04.000 People are using. For instance, these chat bots for things like medical advice, or but what can imagine more competent systems later performing sequential decision-making. 00:36:04.000 --> 00:36:23.000 And you would want to make those safer too. So those might be who exams, if like, for instance, with like Chat gpt plugins, and whatnot that is able to actually interface with the world, and that's creating some some potential hazard. 00:36:23.000 --> 00:36:34.000 So since people are actually using these systems, machine learning systems in the real world there and coming to depend on them increasingly and outsourcing more diseases, making to them. 00:36:34.000 --> 00:36:43.000 I think there are many ways in which a large language models are becoming more safety critical. 00:36:43.000 --> 00:36:51.000 And I know we have sent Chat Gpg. Many times, but that is not a limit to this program. 00:36:51.000 --> 00:36:43.000 Okay. 00:36:43.000 --> 00:36:51.000 So we're not gonna say it anymore. But you understand what we're talking about. So. 00:36:51.000 --> 00:37:00.000 But we remember we're talking about these broad learning enable systems, so large language models are one. 00:37:00.000 --> 00:37:02.000 But there are many different things you can think about in here. 00:37:02.000 --> 00:37:11.000 So keep in mind. We have a broad thing, and actually gets to the next question, does any does a learning and system have to have a physical component? 00:37:11.000 --> 00:37:19.000 Or can it be entirely algorithmic? 00:37:19.000 --> 00:37:26.000 I think there are many relevant learning enabled systems that are not having actuators out indie the real world that are still nonetheless, impacting, impacting it through humans. 00:37:26.000 --> 00:37:38.000 So it doesn't seem strictly necessary. 00:37:38.000 --> 00:37:46.000 Right, and to add to that disability specifically talks about identifying the learning enables system, and that needs to be there. 00:37:46.000 --> 00:37:51.000 But you can have other things around it, like the physical system, or human in the loop. 00:37:51.000 --> 00:38:05.000 So these are all optional things. But what we really need to see is the learning enabled component. 00:38:05.000 --> 00:38:00.000 Hey! 00:38:00.000 --> 00:38:12.000 Thanks very much, and welcome to. We introduced you. Thanks for answering me. 00:38:12.000 --> 00:38:25.000 So does it do. The proposed projects have to demonstrate applicability at the proposed frameworks on more than one exemplar. 00:38:25.000 --> 00:38:26.000 That might depend on how broad the exemplar is. 00:38:26.000 --> 00:38:41.000 But generally there's a prioritization toward having a very general insights that are applicable to lots of different systems as opposed to a narrow, very narrow, specific application. 00:38:41.000 --> 00:38:45.000 Remember in the synergy we're gonna ask for an evaluation. 00:38:45.000 --> 00:38:52.000 So you really do want to think about the broader ideas, how they can be evaluated. 00:38:52.000 --> 00:38:55.000 Yeah, this will also that depend on the definition. Right? 00:38:55.000 --> 00:39:02.000 So what's your claim? Safety, your problem, then? Basically, you have to test it in your domain. 00:39:02.000 --> 00:39:07.000 So, if you only claim one domains, and you can only need to tie them on one domain. 00:39:07.000 --> 00:39:18.000 If you claim this, both generalized approach you'll have to test. But this is the 2 or more verify your system. 00:39:18.000 --> 00:39:27.000 Thank you. Jay. 00:39:27.000 --> 00:39:27.000 For all of you to keep asking me if we're gonna have the slides available. 00:39:27.000 --> 00:39:40.000 The answer is, yes, they will be on the website, and the video will be recorded, and we'll be on the website too. 00:39:40.000 --> 00:39:42.000 Is there notion it? Okay, here's the safety question. 00:39:42.000 --> 00:39:50.000 Is there notion of safety subject to interpretation with respect to the system under consideration, or is there an objective notion? 00:39:50.000 --> 00:39:56.000 We must adhere to. 00:39:56.000 --> 00:40:01.000 It's the answer is you're providing us that notion of safety. 00:40:01.000 --> 00:40:10.000 However, it has to be a notion that shouldn't be such a. 00:40:10.000 --> 00:40:23.000 Notion that no one on the either the panel or program directors really understand should be clear to us. 00:40:23.000 --> 00:40:28.000 Thank you so can you elaborate? There's a question on this. 00:40:28.000 --> 00:40:28.000 And Jay mentioned this earlier, but it's on the security aspects. 00:40:28.000 --> 00:40:37.000 Finally research on securing learning and enabled systems against adversaries is not in scope. 00:40:37.000 --> 00:40:43.000 David, you want to explain what we were, that one. 00:40:43.000 --> 00:40:50.000 I know you mentioned earlier. 00:40:50.000 --> 00:40:51.000 They also meet. 00:40:51.000 --> 00:40:54.000 Yes. 00:40:54.000 --> 00:40:56.000 Yeah, it's, for example. 00:40:56.000 --> 00:40:58.000 So people ask for the for the light worker, security. 00:40:58.000 --> 00:41:02.000 So those problem is not included. Right? 00:41:02.000 --> 00:41:12.000 Also asic problem fairness problem. So that's we have other Nsf program to just address those specific problems. 00:41:12.000 --> 00:41:19.000 So for this question, we focus on the safety. So also, safety here is quite broad. 00:41:19.000 --> 00:41:28.000 So you have defined, and to tell us what's the safety you train English, and then you you can verify it. 00:41:28.000 --> 00:41:37.000 But we not talk about the fairness. We don't talk about network security, and we don't talk about Asc. 00:41:37.000 --> 00:41:37.000 Yeah, yeah. 00:41:37.000 --> 00:41:44.000 We do talk about deployment in very challenging environments and situations, or in the face of size, of some very pessimistic hazards, though. 00:41:44.000 --> 00:41:57.000 But yeah, things like network security or intrusion. Detection is is generally covered in other programs. 00:41:57.000 --> 00:41:57.000 So! 00:41:57.000 --> 00:42:01.000 Yeah, he has. The security is a specific means, is a networking. 00:42:01.000 --> 00:42:05.000 Those kind of a security attack. 00:42:05.000 --> 00:42:18.000 Could it proposal be theoretical study with simulated experiments? 00:42:18.000 --> 00:42:29.000 I think. Yes, the solicitation asks for that could be a theoretical study with experimental validation of the results. Yes. 00:42:29.000 --> 00:42:36.000 Only question there is when you say simulated to me, that becomes more. 00:42:36.000 --> 00:42:53.000 The less the property, less of this synergy element, then the and that first part, first element of the solar station. 00:42:53.000 --> 00:43:04.000 It's hard to argue that experiments on a simulated system are and a meet us. 00:43:04.000 --> 00:43:00.000 So we can do the smaller awards on a completely on a theoretical with simulated experiments. 00:43:00.000 --> 00:43:12.000 Send her, just stick. 00:43:12.000 --> 00:43:16.000 But when we get to the synergy we're really looking for a validation plan for it. 00:43:16.000 --> 00:43:17.000 Right. 00:43:17.000 --> 00:43:29.000 So so somebody was asking, you know, if we're gonna do that, should these systems that are deployed, they said in industry. But it could be just deployed anywhere. 00:43:29.000 --> 00:43:35.000 So if we're talking about simulated systems and simulated data, that's enough for the smaller awards. 00:43:35.000 --> 00:43:47.000 And then what we're talking about is having in the synergy level that these are deployed systems that you're doing. 00:43:47.000 --> 00:43:51.000 Or a subscale representation. 00:43:51.000 --> 00:44:05.000 Are we considering human safety or system, safety or both? 00:44:05.000 --> 00:44:09.000 Well, I think it. The solicitation speaks about safe systems. 00:44:09.000 --> 00:44:10.000 But the way systems interact with humans could introduce safety concerns that could potentially be addressed. 00:44:10.000 --> 00:44:32.000 So, for example, a thing that would be in scope is, if you wanted to talk about, do a project on system honesty like, how can an non-expert human tell if an AI system is telling them false information when the AI system might have more familiarity with that a certain domain like 00:44:32.000 --> 00:44:38.000 coding or medicine than that human might be something that has a human component to it. 00:44:38.000 --> 00:44:48.000 But then the research project would involve like, how can you have guarantees that that system is not going to live? The human. 00:44:48.000 --> 00:44:54.000 And if you're gonna do something like that, make sure you have ground truths for what line to humans do because it medicine. 00:44:54.000 --> 00:44:59.000 There are so many different outcomes that are possible, and finding out what ground truth is. 00:44:59.000 --> 00:45:05.000 Often a challenge, so make sure when you're thinking about it. 00:45:05.000 --> 00:45:09.000 You're thinking that way. Make AI library safe, not crash! 00:45:09.000 --> 00:45:24.000 Is that in scope. 00:45:24.000 --> 00:45:29.000 I'm not sure but my guess would be no, because the libraries are like like code written. 00:45:29.000 --> 00:45:33.000 Maybe 2 train ais, but libraries aren't themselves. 00:45:33.000 --> 00:45:40.000 Learning enabled systems like but the library isn't itself doing learning, or it's like more. 00:45:40.000 --> 00:45:47.000 Just some software somebody has written that might then go interact with learning enabled systems. But. 00:45:47.000 --> 00:45:53.000 Yeah, I think, making a good point, you'd have to argue the safety question. 00:45:53.000 --> 00:45:53.000 And so you might have a library that you feel like makes this. 00:45:53.000 --> 00:45:59.000 But it doesn't seem like it's from the outset when when we're just talking about it. 00:45:59.000 --> 00:46:08.000 Now it seems like it's out of scope. But if you felt like it really was, you can also send us a one page project summary that we can give you feedback on too. 00:46:08.000 --> 00:43:44.000 And so we are encouraging everyone to do that. To send a one page summary. 00:43:44.000 --> 00:44:14.000 Or sub scale, representation. 00:46:18.000 --> 00:46:32.000 When let me also add, though, and you know, if you have trouble under, if you, if you're not convinced that it's safety, think of that! 00:46:32.000 --> 00:46:41.000 It from the panel perspective. Yeah, they're gonna be one of the ultimate judges. 00:46:41.000 --> 00:46:52.000 And if it's not, if you have your doubts, yeah, don't depend on the panel to not have system downs. 00:46:52.000 --> 00:46:58.000 I think that's always a good point. If we can't answer it easily, it's gonna be hard for the panels. 00:46:58.000 --> 00:47:02.000 Exactly. That's exactly my point. 00:47:02.000 --> 00:47:10.000 So one of the questions here is is the solicitation targeting only deep learning components are also on classical learning. 00:47:10.000 --> 00:47:15.000 Components, like networks. 00:47:15.000 --> 00:47:24.000 Yeah, I need any components. So we don't really specifically for any based authority or system. Right? 00:47:24.000 --> 00:47:28.000 So the people can use any technology. You you can even come up with. 00:47:28.000 --> 00:47:28.000 It's a new approach to or design principle that I can. 00:47:28.000 --> 00:47:36.000 I can design the safe safer than mean systems. 00:47:36.000 --> 00:47:34.000 That's the way we find, too. So I started thinking, you can verify it. 00:47:34.000 --> 00:47:43.000 That would be good. 00:47:43.000 --> 00:47:51.000 Though there may be some headwinds if you're trying to study solely support vector machines or something like that. 00:47:51.000 --> 00:48:04.000 And because some of them may have less applicability in many applications, so there might be some tension there in not doing a deep learning systems, there might be less applicability. 00:48:04.000 --> 00:48:09.000 And that might be hard to argue, for its broader impacts. 00:48:09.000 --> 00:48:14.000 There is a question on the process which is when I can answer. 00:48:14.000 --> 00:48:21.000 If somebody says I've never applied with with with a partner agency when it's Nsf. 00:48:21.000 --> 00:48:27.000 And open fill in good ventures, so is it different, and is it different? 00:48:27.000 --> 00:48:37.000 Not a whole lot different, but our colleagues that are here now will also be sitting and serving Pamels, and then, when, as we're making decisions, they will also have. 00:48:37.000 --> 00:48:45.000 Input but remember, this is a joint project. So, and it's live by Nsf, so what it looks like from the inside. 00:48:45.000 --> 00:48:48.000 It'll look like Nsf. 00:48:48.000 --> 00:49:00.000 But in partnership with our colleagues, so. 00:49:00.000 --> 00:49:05.000 So there's a question. Is it limited? What the means of learning and enable systems are interested in the call? 00:49:05.000 --> 00:49:13.000 Is it limited to long, large language models like, or does it come on deep learning based systems? 00:49:13.000 --> 00:49:14.000 I think we just. 00:49:14.000 --> 00:49:14.000 Yeah. We kinda answered that like multimodal systems seem interesting, too. 00:49:14.000 --> 00:49:29.000 And there could be not in deep learning ones that a lot of the a lot of applications are deep learning, and so. 00:49:29.000 --> 00:49:29.000 So, what's the composition? In the review panel for this? 00:49:29.000 --> 00:49:36.000 It will depend on the projects there will be. I don't know. 00:49:36.000 --> 00:49:47.000 And then did David Jay for me, or whoever wants to win. 00:49:47.000 --> 00:50:01.000 Oh, I think it will. It will be. It will compose us people who are researchers who are experts in learning machine learning as well as the social formal techniques and other experimental techniques. 00:50:01.000 --> 00:50:16.000 And implementations of these systems. So it's it will be a mixed file, a paddle with mixedx. 00:50:16.000 --> 00:50:25.000 Okay. And so what's the difference between the requirements between the foundation projects and synergy projects? 00:50:25.000 --> 00:50:35.000 Can 2 researchers from different institutions of life in the same project has Co. Opis. 00:50:35.000 --> 00:50:40.000 Are you simply talking about collaborative proposals? 00:50:40.000 --> 00:50:45.000 I think there's a question about foundations versus Synergies. 00:50:45.000 --> 00:50:49.000 But then there's also a question about I think it's collaborative. 00:50:49.000 --> 00:50:51.000 So yes, you can do collaborative projects. 00:50:51.000 --> 00:50:55.000 Yes, you can do collaborative. And so the other is. 00:50:55.000 --> 00:51:02.000 The other question, and one pi applied for both. 00:51:02.000 --> 00:51:05.000 Yes. 00:51:05.000 --> 00:51:07.000 I think the answer is, yes. 00:51:07.000 --> 00:51:07.000 Bye! 00:51:07.000 --> 00:51:15.000 But that's a that's an imitation that basically each time you can only apply for one category for 1 one proposal in total. 00:51:15.000 --> 00:51:19.000 You will only apply for 2. Right? So that's what the. 00:51:19.000 --> 00:51:27.000 So if you feel pi this time for both categories that you're done so, you cannot apply next time. 00:51:27.000 --> 00:51:34.000 So if you apply for one category this time, you can probably use the category next time, so don't. 00:51:34.000 --> 00:51:42.000 Okay. 00:51:42.000 --> 00:52:02.000 Does safety include consideration of age and contacts, such as what systems are used in an educational center? 00:52:02.000 --> 00:52:23.000 So really it there depends. What? What are you defining as the safety properties that you're going to evaluate within the context of that setting? 00:52:23.000 --> 00:52:32.000 There's also a question of whether fairness which I believe we've answered before or privacy are they? 00:52:32.000 --> 00:52:40.000 Are they safety concepts? 00:52:40.000 --> 00:52:48.000 So, since there are limited things that they could find things like fairness were generally more out of scope. 00:52:48.000 --> 00:52:52.000 But when instring safety more broadly, you know there are obviously many parts. 00:52:52.000 --> 00:52:55.000 It's a broad section. Technical problem involves governance involves security. 00:52:55.000 --> 00:52:58.000 There's a lot of things to make makes us a safe. 00:52:58.000 --> 00:53:10.000 But in the this program some of are getting many of the areas that were neglected in research previously are not covered by other programs. 00:53:10.000 --> 00:53:18.000 So. Hence they're generally not as or it's not really in scope. 00:53:18.000 --> 00:53:26.000 Yeah, so privacy research, many covered by this trustworth computing program already in side. 00:53:26.000 --> 00:53:28.000 So I think that's pretty much out of scope. 00:53:28.000 --> 00:53:32.000 This program. 00:53:32.000 --> 00:53:42.000 It would just be difficult to argue privacy as the primary safety outcome of the project. 00:53:42.000 --> 00:53:48.000 That would. It'd be a tough argument. 00:53:48.000 --> 00:53:53.000 Alright! Dan mentioned, uncertainty, calibration. Can we use? 00:53:53.000 --> 00:54:02.000 That as the only safety criteria in the program, or is it too general? 00:54:02.000 --> 00:54:02.000 That's obviously going to be left up to the panel. 00:54:02.000 --> 00:54:30.000 Maybe one would want some more specific stuff or things to make it have more of a safety flavor you could potentially argue that we're wanting to have certainty about when the models are applicable, or when they or maybe when there's some hazards in 00:54:30.000 --> 00:54:36.000 the environment, or if or if the machine learning systems themselves are creating hazards, and we detect that. 00:54:36.000 --> 00:54:36.000 So might want to add more resolution to exactly what you're going after. 00:54:36.000 --> 00:54:47.000 Instead of using just a a broader buzz word like uncertainty. 00:54:47.000 --> 00:55:02.000 And going off the language in the solicitation to describe the safety properties. In plain English, maybe you would want to draw out what onsafe things might happen as a result of the model being miscalibrated in a particular way. 00:55:02.000 --> 00:55:05.000 Yeah, thank you. There's quite a few questions coming through on. 00:55:05.000 --> 00:55:09.000 Is this related to Hci, why isn't this an Hcc. 00:55:09.000 --> 00:55:18.000 And our human centered computing. And I think one of the questions. 00:55:18.000 --> 00:55:23.000 One of the questions that that comes up here is so. 00:55:23.000 --> 00:55:32.000 This is not Hcr and human centered computing group does lots of works about issues of adverse events and things happening. 00:55:32.000 --> 00:55:35.000 But they're really documenting them, and they're really documenting what's happening. 00:55:35.000 --> 00:55:48.000 This is really to prevent. So we're really trying to create systems here, create tools, create methods and techniques that allow these systems to be safe it isn't documenting that they're not safe. 00:55:48.000 --> 00:55:53.000 It is really trying to address the safety issues upfront. 00:55:53.000 --> 00:56:01.000 So, if you need to do Hci type methods to address safety, then that's in scope. 00:56:01.000 --> 00:56:12.000 But make sure that you're addressing the safety. 00:56:12.000 --> 00:56:19.000 How many, I don't think how many pages are appropriate for describing the components. 00:56:19.000 --> 00:56:34.000 Rationale, safety, plan and validation are these separate sections from the main technical component are included in the technical part. 00:56:34.000 --> 00:56:39.000 Yeah, 15 pages in the project. Just description. To give us a complete picture of what are the safety properties? 00:56:39.000 --> 00:56:54.000 How are you going to but experiments? Are you going to prove it's up to you as a proposer to decide how you allocate? 00:56:54.000 --> 00:57:16.000 That's page space. As you think about this. Also, think about what make what will make it easier for the reviewer to find the information that they're trying to identify from the solicitation don't hide information. 00:57:16.000 --> 00:57:20.000 Make it easy to review. 00:57:20.000 --> 00:57:26.000 Alright, so I hate to say it, everyone but we. We were so busy answering questions. 00:57:26.000 --> 00:57:32.000 We ran out of time. So I did. I just looked down and realized it's 2 Pm. 00:57:32.000 --> 00:57:42.000 And we were still excitedly answering questions. If you are interested in sending in a one page summary, please follow the emails in the solicitation. 00:57:42.000 --> 00:57:51.000 Send a one page. Project summary talking about what you're learning angled system is what your notions are and what you plan to do. 00:57:51.000 --> 00:57:59.000 And so you can get feedback on your projects. I wanna thank open fill and good ventures for being a colleague. 00:57:59.000 --> 00:58:12.000 Here they are, supporting, they are coasting with us and again, as we noted, they'll be observers in the review. Multiple questions about this.