>> we are going to leave it now as lawmakers are holding a hearing to examine software and how it plays a role in innovation. you are watching live coverage on c-span three. >> we expect the systems and code that we rely will work when needed. despite this criticality, we see a department that struggles in peerk operating and

prioritizing software. congress, do know when something is gone and something is right or when something is working or not working. we see the reports, we read the studies, findings that are remarkably consistent from the 1980s to today. we know that some things are fundamentally amiss. my hope for today is for the witnesses to not only describe the problem but contextualize where possible what is being done to date and what the largest barriers have been in a meaningful manner. to help with that task that i'm about to introduce. dr. richard murray, professor of control and dynamics systems and bioengineering at caltech and co-chair of the 2019 defense, innovations, boards of software accusation. and dr. dan pratt at the hudson

institute. thank you for being with us today. i will now recognize the ranking member for his opening remarks. >> thank you, mr. chair. looking forward to hearing from you. the department of defense and u.s. government has been critical in the development of software and technology in this country as a representation of silicon valley. i remember that it was our mission to the moon that led to the acquisition of semiconductors. that is what spawned silicon valley. by semiconductors, you never would have seen the development of silicon valley and the technology. the innovation is happening in my district and the private sector, we need to figure out how we continue and strengthen and private sector recognizing the dynamic nature of software,

how quickly it changes, how we need that innovation to keep us the strongest military and country in the world. i am looking forward to your comments and your suggestions. >> thank you, mr. chairman. >> we will now start with the witnesses. misses lord, you are recognized for five minutes. >> thank you very much for chairing this hearing and thank you members of the --. in an era of strategic competition among technologically advanced powers, software shaped the nature of deterrent and defined national security advantage. the urgency to empower our defense and apparatus across all domains with both the best existing and emerging technology is critical to not

only prefers our freedoms but those of our partners and allies. transformed our commercial sectors. in turn our everyday lives. now, we must harness and apply this ingenuity and innovation to bolster u.s. military superiority in the digital age. given current geopolitical conditions, the stakes could not be hardier. software development and support of our country support and infrastructure needs to meet the challenge of the moment. to fall short now would not just be a bureaucratic debacle but a source of imminent risk to our ability to deter, fight and win. the ability to quickly develop and deliver capability to close the gap between information discovery and mission response is a defining differentiator

and emerging global competition. defense and intelligent agencies must develop, acquire, execute and maintain software to meet current mission needs while also having the agility to quickly respond to future threat environments. the statutory, regulatory and budgetary framework for these agencies are right for streamlining to build and maintain the nation's software advantage. the department of defense, procurement process, is one of the greatest challenges and opportunities to software acquisition. often software is purchased using the same approach that is traditionally employed for major hardware systems. typically this entails setting rigid requirements, lengthy solicitation processes and ultimately coming years later, to adapt software that is often

obsolete upon delivery. although alternative pathways exist, they are only effective as an acquisition professional's implement them. funding professional training and development for acquisition professionals to ensure they have key skills for implementing the full spectrum of acquisition approaches will enable the best and most innovative technology to be quickly provided for our national security workforce. dod must operationalize policies and procedures to support the modern delivery and practices such as agile lifestyle software and service delivery, human centered design, and modern technology stats. training the acquisition

workforce is necessary but not sufficient to modernized development and deployment. resourcing must be available to provide flexible and leadership must demand that all relevant procedures and processes are employed. my submitted testimony goes into more details on these items. i would like to close by acknowledging three efforts that are producing actionable recommendations that might be useful to the subcommittee. one is the commission on planning, programming and execution. we just last week produced our final report. we talked about recommendations that could be employed.

many would help our software initiatives. two, the software defense coalition that is led by jane lee is producing actionable recommendations for the subcommittee. and finally, the atlantic council commission on software defined warfare on which i serve, is again producing actionable recommendations. i urge the committee to follow up on these. thank you. >> thank you, ms. lord. are recognized for five minutes. mr. luttrell and distinguished members. thank you for inviting me to speak on the topic of software development. from 2016 to 2021, i was a member of the board and co- chaired along to do software. this was established in 2018. our report was called software is never done. the key findings of our report was that congress and dod had

been talking about the importance of software. in many ways, the report was just a rephrasing of the 1987 defense task force on military software 32 years earlier. identified itself over previous on that topic. chapter three, been there, said that, we know what we need to do. we need to figure out how to actually do it and get to it. our 2019 report, i believe we are still the most important points. the first is that speed and cycle time are the most important metrics for software. being able to deploy faster than our adversaries means that we can provide more advanced and be more responsive to end- users. and gives us a tactical advantage on the battlefield by allowing operation and response inside our adversaries. second, software is by people and for people.

dod resource policies are not conducive to attracting and promoting digital towns. talented developers and acquisition personnel are often put in jobs that do not allow them to make use of their talents. particularly in the military were job assignments may not recognize the importance. today and dod, the people with the necessary skills exist but instead of taking advantage of the skills, we often put them in environments where it is difficult for them. third, software is different than hardware and not all software the same. thomas and dod have established instructions that govern the development, procurement and sustainment of defense systems. software development is fundamentally different. software should be developed and continuously improved using much different cycle times and maintenance strategies. software is never done and

mostly managed treated differently than hardware. i took the opportunity to read reports on the implementation of some of the recommendations for the study we have partnered with. i was pleased to see that congress and dod have made progress in implementing many of our recommendations including establishing pathways for software and exploring new procreation categories. these are important steps and they should be continued and accelerated. in addition to these important actions focused on the acquisition process, dod implemented many actions on primary and secondary recommendations. some of the most important providing guidance including reciprocity and continuous adls. as of april 2023, guidance for continuous adls has not been processed. dod reports progress but

appears they have let to establish. a well-defined software developer including service members will out dod to retain to design, build software systems. finally, an area that is completely different when we did the study is the role of artificial intelligence and military systems. the implications of ai will be profound across all areas of society. the field is changing so rapidly it's impossible to predict how ai is working progress. ai is poised to revolutionize the way we right, test and deploy. already riding code based on the speed development. in the future, it will be integral. these developments will democratize development and ways to do it drastically

reduced time. at the same time, software is using for military systems is critical to fail so we must find ways to harness those. dod must stay on top of these developments and take advantage of the current u.s. leadership being development. congress working with dod plays an essential role in breaking us out of the cycle. thank you for your attention and i look forward to the session. >> thank you, dr. murray. dr. pratt. five minutes. >> thank you for inviting me here to speak on such an important topic. i'm here an individual capacity . i serve in diverse roles offering perspective on technology and threats. broad techno-economic shifts

and the vibrant ecosystem. as you all know, software is ubiquitous. powerful implications for economic productivity and government effectiveness for cybersecurity and the character of national security. these technology goal changes are overlaid on the context of our time. strategic consultation. in the crux of the problem for national security is this. and a sustained competition and the long-term competition, advantage ultimately depends on the ability for one side to adapt. year-by-year mitigating weaknesses and advantage. the kind of questions we ended up with are things like our weapons systems, relevant against a relentless pace of threats. can we invent new ways of fighting that put the prc on the back foot.

these are the issues that the department of defense must tackle if it wants to compete. everyone of these issues now depends on software. even changing i military units' tactic. we need look no further than the battlefields of ukraine to find evidence that units which are able to change the software more quickly see better outcomes. we face a choice. we can be victims of software, cursing its bugs and overruns. or we can harness it for competitive advantage by leveraging american ingenuity, a robust talent base leading technology. that is our question. can we create a defense system built for evolution and adaptation? my central message is this and it will echo those of my fellow

witnesses here. the process of getting code from a programmer to an operational system is critical. we blur the lines between what is development, building something. and what is operations, using that same thing. making this quick and robust is a necessary for competition. they remain in the minority and they faced daily struggles against organizations and processes built for another era. i call your attention to two axonal items. as we've discussed where adls how the department decides that software is safe to deploy and use. the second item, towns. attacking on the first topic i

will introduce an analogy. in many ways, making software resembles molding wet clay. forming it into some finished piece. when an engineering team works with source code, they can quickly adjusted, make fixes. once the code is compiled, built, shipped, it becomes fixed and brittle. the code only works on one particular type of processor. what we find is modern software so complex, you need to feed the results back to the engineering team. unfortunately, our 312 process makes this quite difficult. on the second topic, technical talent come a software is a complex and technical subject matter. details matter. we hear this quick big headline sometimes. one simple trick can solve dod

software. one software factory to rule them all. it's all about agile. those are all great tools. those are useful things. but navigating that complexity requires judgment and organic technical talent on the part of the department. the dod needs leaders driven by mission. it doesn't need armies of coders. they can attract this talent if given the right tools using things like appointments and giving these people autonomy to make a mission impact. thank you. >> thank you, dr. patt. i will move it to questioning. authority coming all three had on that. i hate to say it, it seems, you guys shouldn't leave. i'm going to talk to guys if you want to hang out. yuck. totally cool. sit back down. the authority -- i don't -- it

seems like it's the improper medium for software innovation. software is updated every half second of every second of every minute of every day. it moves so fluidly. for the past 40 years, the authority seems to be bogging the system down. i'm asking three subject matter experts. we'll start with you mr. patt, or ms. lord, we will start with you. ladies first. what is the fixed? i don't want to report that we will never read. what is the fix to fix this problem? this is the way we lean in front of our adversaries. the system is failing itself. >> the challenge is the need for speed. i believe there are two things that we need to do. one, moved to continuous adls. there's been a lot written and a lot discussed. continuous atos are not yet

implemented. this would be a very good thing to ask a dod leadership. secondly, we are repeatedly across even programs, military services, agencies, not allowing reciprocal rights for ato. the same software is being reauthorized again and again. those are the two key things that i think you should push on. absolutely not looking at discrete, repetitive, approvals. right now come in fact, there is only a requirement to approve 12 systems a year, which given the fact that most of our systems went on hardware, software and data is frightening. >> dr. murray. >> i completely agree. we think about continuous ato in particular. we need to be able to stay, this software needs to be updated. the longer we wait by not giving them the code they need

to do. how do we get to the point where we are an industry, right? if a zero day comes out, some attack on my cell phone, there will be an update in the next day or two, right? they've already figured out, it's going to satisfies our own . we need to find what that is. i think you should be asking on every program, what is the cycle time is going to the software. how much is that is the ato process. if that ato, if that more than a day, there's a problem, right? there needs to be a continuous ato where we automatically check, does a satisfied? this is important code. we need to be able to get out there quickly. >> dr. patt. >> i agree with those comments. the ato is about the risk of using the software. the schedule along with mission

risk. if it's buying body armor, you can separate these issues. is it safe to use this on a mission. does this help support the mission. these lines get blurred. i will say that one of the places where you see progress in the department is where they try to put these risks together. if you look at, in the navy, there are program offices and program managers which both own the risk of use of the software on the operational network and the development of software. when you move those things together, you tend to get these were mission focused outcomes. these are organizations which have been able to use continuous ato is. i would expect that same characteristic to carry us forward. as dr. murray said, we often focus on adls. we forget the risk of not updating the software, of not

deploying new features. we become too focused on compliance, checking the boxes of, you know, did we meet the boxes we said we did would? we forget the underlying problems. the most important thing is the cycle time that we keep coming back to. you can keep updating and mitigating problems. >> if i may have one quick follow-up on that, i think we need to differentiate in the department between risk management and risk elimination. we are never going to eliminate all risk. these trade-offs are what take human judgment calls. what dr. patt is talking about. i think it's very important for congress to recognize those individuals in the department who are leaning forward and

demonstrating, embracing authorities, policies, procedures. i would say who was, has done that admirably. >> thank you, ms. lord. for all the young men and women, they are talking to you. you are the next generation that will keep this country ahead across the globe. i highly recommend you play this takeback because we are counting on you. do you understand? outstanding. your recognized for five minutes. >> in my district, when you have a software challenge, they don't just go out and buy new software. in fact that the last thing they do. they have a mission and they figure things out.

i've heard that they just buy new software to check off the box that they have complied. this is innovating in the way that the private sector does. could you comment on how we change that culture other than getting tim cook to run these things? >> i think it comes down to investing in the human capital. if individuals are not trained to be smart buyers and we cannot attract contemporary coders and so forth, we cannot change that culture. right now, we have a huge issue. this is mentioned in our final report with modernizing a lot of our business systems. we are pretty good at talking about from a war technology point of view.

we typically do not talk about innovating business systems. we need to digitize those systems and we have to attract individuals who want to work on contemporary systems. right now, we haven't hard time attracting a lot of software specialist in the department because we are working on 10, 20, 30-year-old software. frankly, there's not the knowledge as to what can be done. i suggest working with dau, training the professionals on what a digital environment really is. thank you. >> i would say, i completely agree with you. we can say this offer is not doing what it needs to do so let's go specify and put a bunch of requirements to find a new piece of software that is what you do that and go by that. we need a platform. that platform is an enduring capability. we are going to be delivering apps to cell phones forever. we are going to be tracking satellites in space forever.

there software that going to do that. that platform is something that is going to get updated. we are not going to say, right down the requirements for that and then it is done. it needs to be something we say every year we are going to invest a certain amount of money that going to do that. we need to create a system in which we think of software as an enduring capability. to get into that mind-set, we seen in the commercial sector. we are just able to jump on all those things. we have to do that within the dod. i think there are spots that are doing that. ms. lord said we need to recognize those. i think that we see some of that starting in the software position pathway but it's for things that are pure software. so much software is sitting on top of hardware and the hardware mind-set dominates. we got to break out of that.

that software also needs to be upgraded. some of that software should stay there for a decade or more but much of that software, things that should really be updated on a regular basis. >> i will give you the chance to answer it. if there is a company and they are awarded a second phase, a lot of the startups, they often are waiting for a year and a half to get the sbi. is there a way you could authorize the contracting officer to at least get a letter of intent to possibly that they are going to get the grant? the startups don't make it. >> yeah come on this question, there are a variety of tools the government can use however there are also limits. a frequent limit that comes up, often you have to wait for funding to become available so that the government can't just

indicate advanced notice. some of the things that they pbb has done could create additional flexibilities which could be delegated down. which could allow flexibility, allowing the same year, flexing of priorities of being able to move some money that isn't being put to good purpose to advancing, say, a phase award. >> following up on that a little bit, i think it's important to note that right now the national academies is running a study on recertification on the process for just this very reason. as dan mentioned, this report talked about how if we didn't have so many discrete budget line elements but had more capability elements, we could allow the peo owes to have more agility especially in the year

of execution to move money towards emerging technologies and the smaller companies. right now, unfortunately in my opinion the process as an afterthought for most peo owes and pms. it's a little bit trickier to use. again educating the workforce is critical. >> mr. gates, you are recognize for 30 minutes. >> reported that the tr three software delay forced the dod to combat code f-35s. i don't know what it means to combat code something. it sounds dangerous. maybe we could draw on your expense and you can explain what combat coding is. >> i'm not sure i totally know the definition and sounds like coding on the go. i will say is the perfect example of a program that

looked to our major critical for our national security. frankly to a large degree has excluded the small companies where the predominance takes place. that is where the coding capability is. having lived through the tr 3 drama, we need to decompose this and make sure when we are awarding these large contracts, that we are reaching out to the companies who know how to do this coding and again, manage risks, don't eliminate risks and find a way to bring them in. >> here's the challenge we have with a lot of these companies. they are dying to get the contract. they do also to things to get these spee11s. and then they want to protect their source code to access it to kind of springboard off of it.

is the f-35 one of these systems where the upgrades that are needed are now being executed through this combat coding system? >> i don't know whether or not the system is being used there but it certainly the case that it is a large program used from what i can tell, a very traditional process. >> you mean that as a critique? >> i agree. again, some components is not something you should be changing every day. other things are that we want to be agile about. how do we end those programs, not that we should break the rules because now there is a crisis. but rather, we have a well- defined way of putting software in there and doing continuous atos. what is the frequency on which we are going to upgrade? >> what they would love to say to us is, you need to upgrade

it frequently. of course, we can't share our code with anyone else so they might be able to participate in the update. how i braked through that? >> a great question. i will answer little bit about it. i think that you have to think about where these pieces of software come from. >> will have to wake up keith martin with smelling salts. not every update has to come through. >> let me just add some operational context to this. sometimes i think that in time, it's hard to understand how important delivering software updates is. if you just look at the conflicts playing out, ukrainian radios only last about three weeks before some countermeasure comes along and you have to reprogram or change the form. we see the excalibur targeting system dropped from 70% effectiveness to 6% effectiveness over a matter of a few months. if we expect our major weapon system programs to follow these

processes or spend years planning a refresh, how you ever hope to adapt in competition? >> that what we are doing now. >> that right. this is the problem. >> much of it is going to go back. there are two principles. when we talked a little bit about. the other principal you just brought up is this notion of vendor lock. i want to point out that an ato is another mechanism by which one can achieve. some companies get a continuous ato and then they seek to use that to lock themselves and because it is so expensive. companies can spend $1 million getting through an ato process. there are a set of principles and these are known as the intervals. congress put this in place. ms. lord oversaw a number of efforts. what most principals do is they

say, the program manager in charge of the acquisition shall define modules and interfaces and make those interfaces available to qualified contractors, to the government. what this does is this allows other companies to come along, they see the data, they see the interface and they can make new software that works for that. the government can figure out how to test this and make sure that it works effectively in context. the principles exist. they are just poorly adopted. >> that because people are protecting their turf. i have further questions about ai but i see my time has expired. thank you. >> you are recognized for five minutes. >> thank you. thanks for being here. i was at the pentagon for seven years. in the 5s years that i've been here, we have done a ton of hearings. it just our cultural problem. speeding up decision-making,

particularly as it relates to tech. to the point where i feel like to use a pentagon term, we are admiring the problem. we love hearing about it. and actually, there is depart his eyes and agreement to speed up that strength ongoing from concept to fielding of technology of any kind. it has led me to believe even as someone who cares deeply about the pentagon and our national security that it is not anyone program or anyone authority. it's really a culture. culture cross administration, culture across leaders. i'm not a burn it down kind of person saying the system doesn't work so we need to destroy it and start from zero. but separating yourself from

the tech, if you could make, some of you have served inside the pentagon, if you could be cleaner for the day, on one cultural change, what would that be? and i would say, aware of the fact that the pentagon has a lot of rules and restrictions when they use taxpayer dollars. everyone else doesn't have to adhere to. the responsibility of stewarding those dollars is different than a small startup in palo alto. it's never going to be the same thing. comparing the two is apples to oranges. speaking from culture as people who have been either in the inside or adjacent, what is the one thing you would change starting with ms. lord? >> will i would implement leadership to demonstrate that smart, risk management benefits

the nation. their four, i believe there are many more motivations and rewards that can be given to those individuals whether civilians are in uniform who lean forward and take smart risks. they manage risk, they don't try to eliminate it. and they move at the speed of need here. and i believe that leadership needs to speak up and make rewards and highlight those individuals who are doing the right thing. recognize that people will falter and not to blame them, if you will, when everybody is looking for a -- >> when i have the chance to visit, people wear their mistakes as a badge of honor. i started the company, it didn't succeed but i learned these important lessons and now i am on the third company.

it was so strange to visit silicon valley and hear people talking about their failures. you would never talk about your failures. >> with all due respect, congress has a piece of this. >> i think is a great question. changing that cultures want to be hard. if i could be going for day, i would change the ways we think about the teams of people working on software. you see a lot of this turnover. people are not there for 10, 20, 30 years. they come in, they start out another company. how do we take advantage, put them in a position that is skipping a couple of levels and other things because they're supertalented, they know what to do. i think they will help change that culture. why are we doing this way? i know how to do it that way. we have to change the culture. we are going to change the culture by saying change. we do that by bringing people

and who think of things differently. >> fostering a culture of doers is really the key here which is encouraging the accountability down to an individual program manager for the outcomes they achieve. you only serve in a position for a few years, it's very hard to have a sense of accountability or pride. the great thing about software as you can compress these delivery timelines and suddenly, you can start to celebrate this. combined with the suggestion that dr. murray just made of bringing people for short tours of duty, i think you can get a vibrancy in the perspective and talent to drive results. >> i yelled back. >> thank you. mr. mccormick, your recognized for five minutes, sir. >> thank you mr. chair. and thank you to the witnesses for being here today.

i love the variety of questions all over the map. i'm going to go a little off. we have a couple doctors and technological fields. i'm a doctor of medicine. it's interesting to walk how ai has been developed and how it is taking is very similar. this is fascinating for the kids up there and how ai is developed. different parts of your brain affect different parts of your body. it's designed for this part of your brain, this part for your egg, this part for your hearing. this is your decision-making, your personality. we have done the same thing in ai since we can no longer shrink the size of these chips. one atomic structure of thinness per layer, which is fascinating to me that we can

actually design software to go into hardware and fit in these capacities to do specific functions like we do in our brains. after all this technology and all the signs come out we went back to the way god divined the brain. this is for mike to technology eskridge. as we do this and we are designing software to match the hardware, ai accelerate that process. how does that affect cyber security, how do we unleash that potential as we design software going forward? >> i'll give it a start. as i said earlier, i think it is hard for us to predict the impact that ai is going to have. it's clear it is going to be huge and we have to figure out how to take advantage.

our adversaries are going to take advantage of that. we are in trouble. one of the things we have to do is start saying, we need to be using ai to design software. we need to be using ai to attack software internal to what we are doing, right? where is that happening now? we right code with ai. we are not going to be able to take advantage because that technology go so quickly. as fascinating because ai is one of those areas, the developers of the technology were legitimately surprised by what it could do. they didn't know it would be able to do these things. it's amazing to do that. he did what it designed it to do. i think it's going to be huge. i think we ought to be asking, where are the efforts that are happening? >> one of the things i also want you to consider, georgia tech has -- we also had 250,000 students from india just one

year studying. we had a ton of kids from china studying. we have production over in taiwan, right? we have production overseas, our technology going back overseas. how do we protect this stuff from being used against us? we are trading this people and producing the chips overseas and a nation we are going to take over. i'm concerned about the security measure. >> an important question. complexity of our software and hardware end design is absolutely exploding. i will say that you can use this complexity as a tool. one of the things you can do is you can create a check, you can produce the hardware and you can set features of it later. you can defer some of that. you can build and forms of defense. at the same time, i don't think it is possible to prevent all

vulnerabilities before hand. this is one of the reasons why being able to go back and drive updates, detect problems and drive updates is so important. we begin to see this now, for example, some of the navy operational networks automated anomaly detection. they are able to push a patch before any users see a problem. to the advantage of the u.s. is how we stay ahead. >> ms. lord, i went to place the question back on you again. we always talk about securing our software. how do we protect our technologies from the bad guys? >> i think what we have to do is look at these from both a defensive and offensive point of view. it's absolutely critical that we have standards that are held to when we look at the system

level, that we also understand the providence of all half where it is software that we include. right now, we don't always understand the beneficial ownership of some of the companies in our supply chain. frankly, ai has done an incredible job just scraping publicly available information to really understand where items begin. the department of defense has begun to put some contracts in place to use that technology. cyber security standards are already to some degree in the acquisition policies. that means to be followed up on. >> mr. ryan, five minutes. >> thank you all for being here. i really appreciate your input and also your service and many various have to the country. despite my desire to ask about tiktok, i'm not going to do that this morning. i'm sure many of our audience

would be curious, myself included. i want to follow-up on something specifically you said, ms. lord. you started to say congress has a role in this discussion about culture and changing the risk, appetite, if you will. could you talk a little bit more about that, any specific ideas you have? >> absolutely. i think often, it is easy to look at what has gone wrong in dod. and pound on that a little bit, if you will. i think perhaps if thoughtful questions about what did you learn from this, how did it happen, how do we do things differently in the future, that line of discussion would be very useful. also, i think asking questions about what is holding you back from taking more risks.

i think there is an incredible fear of failure. if you could engage senior leadership in the building relative to discussions about that and ask them how you motivate and reward them to act more like a commercial sector where we do things quickly. we pull the patches in a day or so. i think that would be really useful. >> setting up others to answer too. not to overly formalize it but do you think you could ever make sense for it to be more public or broad assessment of which programs, which leaders are accepting risks in a more formal way. >> for years when i was in the building, we talked about holding hearings to showcase these types of individuals. i believe the reality is that scheduling does not allow you to do that type of thing. things like tiktok they need to

talk about and work on. there's so much you can do just with recognition with letters, talking to the media, inviting people up to your office for an hour or so. it doesn't cost much but will go a long way. it would cost a very long shadow. >> thank you. >> i think congress does play a role in the way you asked the questions. if you do something, you are forcing us into a system on which we are checking on the boxes. you look at speed and cycle time of our main metrics. how quickly can you get something? can we get a leaderboard going? this program, why is that they are doing better? we've got the best cycle time. we make those things public? here's how quickly these different programs can get things out into the field. >> thank you.

>> one of the most remarkable technological achievements is just how safe civil air travel is. it's remarkable how few fatalities came out of this. the reason it became so safe and so successful is there was a strong culture around a deep reef. if you step forward and you talk about the problem, it's about learning. it's not a bad failure. this is the same principle we need to apply here as we figure out collectively how to do software right. many of you have heard -- many of you have also heard about the air force's management system in the air force general processes. both of those efforts are efforts which dig introspection about their early efforts. they said, you know it's, we

need to pivot. how to approach this, how we think about managing this effort and we need to pivot. those are the kinds of people, the kinds of decisions and that blame free culture that we need to make this work. >> i appreciate that. with the 22nd i have left, recognizing that that has to start from the very very top. it has to be a principle of all the leadership. thank you, i yield back. >> mr. khanna, closing statement. >> i want to thank the excellent witnesses and i appreciate your time. >> thank you again. shedding light and playing information, we learn more from our failures than we ever will

our successes. in the military, we never talked about how well the operation went. we always talked about how horrible the next things were. i've been sitting back here, i know the reason why we are so risk adverse in the house of representatives. i'll keep that to myself since i'm on tv right now. my colleagues and i will definitely discuss that. members will have five business days. this committee is now adjourned. had

a pregame. >> [ inaudible ]

>> it is 100% up to what you want to do. if you want to lead and innovate, whatever that looks like, no one is going to help you do it. we are counting on you to do it. is that a fair statement? go forth and do great things. if you want to be really good looking, become an atheist.