Lessons learned from establishing the Systems Engineering Research Center, a networked University Affiliated Research Center

A University Affiliated Research Center (UARC) is a highly valued engineering, research, and development resource to the federal government, largely within the US Department of Defense (DoD). Only 15 exist, all under special authority of the US Code. The Systems Engineering Research Center (SERC), established in 2008, was the first UARC established as a networked research center, gradually growing to include more than two dozen university members, each offering systems engineering research capabilities. This networked model for a UARC has found support within the DoD—exemplified by the newest UARC established recently by the US Air Force. A large diverse pool of faculty and student researchers arguably delivers the most consequential academic research in systems engineering anywhere in the world. In its first few years, the SERC had to overcome challenges that single‐university UARCs did not face in exchange for the advantages that such a sizable network of universities offers. This review paper explores those challenges, offering lessons for new network‐based UARCs and other network‐based university research centers. Systems engineering communities, globally, are exploring such constructs, and this reflective paper may offer useful insights.

The National Security Agency (NSA) and the Office of the Secretary of Defense (OSD) were the initial sponsors for the SERC, each committed to contributing $1 million annually for research.No formal program or budget line was identified to manage the SERC within the DoD budget.Management of the SERC, and the execution of its research portfolio, was to be performed using each partner academic institution's facilities and administrative (F&A) overhead funding.
After 15 years of operation, the strengths of the networked approach have become evident.The SERC is vibrant, well-regarded across the DoD, the defense industrial base, systems engineering profession, and the academic community, with annual expenditures exceeding $25 million.The greater impact potential of the SERC was realized because of this networked research center model.The UARC breadth and collaborative nature enabled the DoD to not only solve challenging systems research problems, but also improve the systems engineering discipline which is so critical to complex systems issues facing defense as well as national and economic sectors, and foster an increase in the future systems engineering workforce talent that can be accessed by the DoD.

INSIGHTS AND PRINCIPLES TO GUIDE A NETWORKED MULTI-UNIVERSITY RESEARCH CENTER
As we approach the SERC's 15th year anniversary as a networked national research center, some of us who have had a leadership role in its establishment have reflected on the primary challenges and opportunities, and perhaps principles, that were critical and worth sharing

2.1
Research network-faculty across the university network are the critical asset The primary value proposition and strength of the SERC derives from its network of research faculty across the universities that constitute the SERC.This value was emphasized in the proposal that Stevens Building relationships among the faculty in the SERC network, across universities, took time, was based on trust, a shared vision, and set of operating principles.For the SERC, the vision "The Networked National Resource to further systems research and its impact on issues of national and global significance" was galvanizing.The Vision contained the critical elements of focus and success: • Networked: this is stronger than a collaboration, this is a network which forms a vibrant, self-sustaining ecosystem, where the whole is fair greater than the sum of the parts.
• The National Resource: this is "the GO-TO" place for systems research.
• Systems research: this goes beyond traditional Systems Engineering, Engineering Systems, Enterprise Architecture, and extends to systems large and small including the non-technical aspects as well.
• Impact: success is measured by demonstrable impact, not just by publications and tools.The SERC will play an active role in providing the path to impact.
• Issues of National & Global significance: we are after the big problems here, the ones that make a huge difference in our quality of life.
The Vision provided the foundation for the SERC's Operational Principles which enabled the members of the collaboration to work with a consistent value system: • Do innovative, high-impact research • Only perform tasks which are research oriented (usually publishable if not classified).
• Focus research efforts on systems which can be generalized beyond a given domain and transform the discipline.
• Focus on research efforts that have the potential of increasing the security and prosperity of the Nation.
• Focus on research which addresses future needs.
• Don't stop at proof-of-principle prototypes • Work to ensure that there is a path from research results to impact.

• Grow and exploit the research network
• Ensure that the research is conducted by the best available resources.
• Bring in new collaborators who provide long-term strategic benefit.
• Focus on creating a network of academics, industry, and government that is sustainable.

Develop and maintain a research strategy
The SERC was tasked to move quickly, leverage its collaborators, and deliver value to its sponsors.Because the collaboration involved so many universities and researchers, efforts began to represent a collection of projects individually important to customers, but missing the opportunity for collective impact or strategic direction.Therefore, it was critical early in the evolution of the SERC to develop a technical focus and to build agreement among the stakeholders by establishing a research strategy.

Sponsor engagement
One of the early imperatives within the SERC involved nurturing  In parallel, the SERC collaborated with the primary sponsors within the DoD to establish a process for the universities to submit and respond to request for proposals; incubation opportunities; unso- Having some tasks at these higher levels of classification has given the sponsors options to conduct highly applied research with "real-world" data.

Collaborative business practices
As steward of a research network, Stevens developed financial incentives to enhance collaboration and publicized them with research leaders.For example, Stevens reduced or eliminated F&A (Facilities and Administration or Indirect) charges on small tasks and capped F&A charges on large tasks.These actions demonstrated to the collaborators how serious Stevens was in building a strong collaboration and provided an additional incentive to potential sponsors to fund projects through the SERC.
As the SERC became more successful in finding new sponsors, OSD became concerned that more and more sponsor funding would go to support research center management and administration rather than actual research.In response, the sponsor created a special program management task that separated fixed and variable costs for running the SERC.The program management task covered all the key fixed costs, which has significantly reduced the "overhead" for organizations looking to sponsor research at the SERC.Keeping fees low encourages more potential sponsors to participate in the SERC.
Furthermore, a regular cadence of meetings with the contracting officers has helped the SERC leadership convey the unique nature of the SERC, its focus on systems research, its governance and unique contracting arrangements.This has allowed the SERC, as an example, to engage with sponsors beyond the DoD, such as the Department of Homeland Security (DHS) and NASA.

Ensure timely record keeping of operations and impact
Record keeping can be a challenge, even within a single university.Yet, it is vital for a UARC to have the data necessary to support its 5-year comprehensive review.The review begins with the sponsor's request that the lead university provide a report on what it accomplished over the review period, how it managed itself, and how its efforts served the sponsor(s).It importantly contemplates the strategic impact that the UARC is having, and any adjustments to mission or competency needs.As one can imagine, given the diversity of tasks, activities and initiatives within the SERC, this report requires a lot of data accumulated over a 5-year period to robustly represent its impact and influence.
Initially, collecting the required data was a herculean task, requiring data calls to the 20+ universities and its ultimate synthesis by the SERC staff at Stevens.These reporting activities have become streamlined over the years.Today, the SERC has an annual cadence of publishing an Annual Operations Summary Report and a Research Transition Report.This regular cadence has greatly simplified and streamlined the data collection and synthesis process needed for the comprehensive review and subsequent contract renewal.

CONCLUDING REMARKS
The impact of the SERC on the discipline of systems engineering has in 1942 to support the war effort and remains very successful 81 years later.There are now 15 UARCs, each with a different technical focus.A new UARC is typically established through a competition and sponsored by a DoD component such as the US Navy, US Army, US Air Force, or the Office of the Secretary of Defense.UARCs are "trusted agents" of the government, operating with a renewable sole source contract but subject to a comprehensive review every 5 years F I G U R E 1 The Systems Engineering Research Center-university network.to reconfirm the continued strategic need for the UARC and the satisfactory performance of the university leading the UARC.Until 2008, UARCs were run by a single university 1 when the DoD decided to run a competition for a UARC focused on systems engineering research.The DoD's market analysis showed it was unlikely that a single university or even a handful of universities had the full suite of competencies needed to meet the DoD's research needs.The DoD encouraged bidders to submit proposals as teams.Stevens Institute of Technology (Stevens) won the competition as the lead of a team of 18 universities, with the University of Southern California as a special partner to Stevens.The current university network within the SERC is reflected in Figure 1.
The two most recent launches of DoD research centers are both based on the networked-center model patterned after the SERC: the 2021 launch of OSD's Acquisition Innovation Research Center (AIRC) and the January 2023 announcement of a US Air Force (USAF)sponsored UARC focused on tactical autonomy-RITA-Research Institute for Tactical Autonomy.Two of the authors of this paper presented the lessons encapsulated within this paper to the USAF leadership and relevant university representatives to support their intent to establish RITA as a networked research center; and one of them has been invited to serve on the Board of Advisors of RITA-to support evolution of the necessary governance and engagement mechanisms to enable a successful networked multi-university research center.
with the global systems engineering research community, as they contemplate establishing other similarly fashioned UARCs and research centers.Therein lies the focus of this review paper.The paper is organized around the following critical insights and principles 1. Research network: Invest in the synthesis of a collaborative network of universities and faculty-focus on developing trust; 2. Research strategy and focus: Ensure a coherent research strategyfocus on engaging relevant thought leaders in the field; 3. Sponsor engagement: Ensure robust sponsor engagement-focus on research transition and impact; 4. Collaborative business practices: Reduce the business impedance mismatch factors and single points of failure to the maximum extent possible; 5. Reports and results: Ensure documentation of research results, operations, and outreach.
submitted during the competition to establish a UARC focused on systems research.The first two SERC research projects, identified in the proposal, included faculty from five collaborating universities.Additionally, a senior faculty member was identified as the key technical lead for each collaborating university.Within the first year, the SERC formed a Research Council with representatives from the collaborating universities to help inform and drive the research and technical strategy.This new construct was established to bring about a dozen senior faculty, representing the key and diverse dimensions related to the notion of systems research, from across the SERC to be a part of a regular cadence of customer engagement and research strategy formulation.Accordingly, the university network was engaged in the governance aspects of the SERC-the role of the Chief Scientist was expanded to include the role of Chair of the Research Council.This has been one of the defining actions in helping the SERC to robustly leverage the network of universities while building a sense of "ownership" of the SERC across this network (Professor Barry Boehm from USC was the first Chief Scientist and Chair of the Research Council; followed by Professor Dan DeLaurentis from Purdue).The research council has evolved and grown as the SERC has grown with representatives from other collaborating universities and is a centerpiece of today's collaboration, meeting every month.Furthermore, an Annual Research Review event was established during the first year-it is now called the SERC Sponsor Research Review (SSRR).The first Annual Research Review included many faculty from across the collaborators as well as from the sponsors.It was held just 2 months after the establishment of the SERC so that research faculty and sponsors could meet and build the network.It was held at Penn State, not Stevens, to reinforce the collaborative spirit.The SSRR has now grown to attract several hundred attendees and is a major annual academic event in the systems engineering practice and research landscape.It is held in the Washington DC Metro area annually in the mid-November timeframe, allowing for easier attendance by various sponsor communities within the DoD, and more generally across all the US Federal Agencies.The original collaboration quickly expanded to include three new universities that brought significant research capability to the SERC: Georgia Tech, a competitor for the SERC award, along with Naval Post-graduate School and Purdue University.The network further expanded in 2011 to its present size of 22 universities, and again in 2019 to adjust the network based on changing university programs and critical faculty.This ability to periodically adjust the university partnerships ensures a productive and dynamic network, as faculty frequently move to other programs and research interests.In 2023 the network will expand to 24 universities to accommodate additional AIRC focus areas in law, business, economics, and policy.Early on, to again demonstrate the collaborative nature of the SERC, several research projects were launched with faculty leaders not from Stevens.That practice continues today.Most SERC projects are led by faculty from universities other than Stevens, and many do not include any Stevens faculty.Despite the wealth of possible contributions across collaborating universities, early on, research task proposals often came from the same set of prominent researcher faculty.The research sponsors understood the challenge and helped establish multi-year research tasks across multiple universities to help build relationships across the universities and attract the attention of other academic departments and faculty within the SERC universities.

•
Educate the next generation • Provide a focus on education and training research, both in research (graduate students) and practitioners.Often, there are few incentives within universities for researchers from different academic departments to work together, let alone work with researchers from other universities.In the case of the SERC, it took about 3 years for researchers at the different universities to understand how their strengths complement each other and how teaming would deliver more impactful research and value to the sponsor.To help build these relationships, the sponsor clearly stated that they expected multiple universities to work together on large task orders.A regular schedule of research reviews, research workshops and other face-to-face faculty meetings were essential to building those relationships and earning trust.Carefully curated research workshops, often in collaboration with other US Federal Government research and development entities, are a critical pathway to build collaborations within the SERC and beyond.This began early during the establishment of the SERC and continues to this day.Figure 2 shows the profile of two recently concluded research workshops, one in collaboration with the Software Engineering Institute (SEI) at Carnegie Mellon University, and the other with MITRE-both these entities are examples of FFRDCs-Federally Funded Research and Development Centers, supporting a number of Federal Agencies in the US.
broad and consistent sponsor engagement and communications by the various faculty across the collaborating universities.Faculty and universities that do this consistently tend to attract the most research funding.The SERC faculty leaders at Stevens encouraged these efforts by visiting research leaders at the collaborating universities, and supporting them with their sponsor outreach and engagement efforts.Timely and focused research workshops also helped develop a set of engaged sponsors.There are many benefits to having a set of engaged sponsors across the DoD elements.Early on, the SERC "champion" within the NSA took on another role with the agency.The other primary sponsor, the Office of the Secretary of Defense (OSD), assumed full leadership for the SERC and eventually established a line in the DoD budget for ongoing SERC sponsorship.OSD became an active advocate for other DoD organizations to sponsor SERC research.With the newly established core funding line, OSD was able to fund the Service labs to experiment with the methods, processes and tools resulting from the SERC projects.This investment led to customers investing matching funds and follow-on project investments as they got to know the researchers and began to see the value of the research.Collaborations with National Industrial and Academic forums also helped gain visibility and inform research.The SERC created research tracks at relevant conferences and symposia, and participated in working groups and activities that were addressing shared challenges.This constant and robust engagement has paid dividends in the long term.

F
Carefully curated research workshops are key to developing a collaborative culture.Provided here are two examples from 2023.F I G U R E 3 The SERC research strategy thematic areas.F I G U R E 4 Multiple pathways for research engagement.The sponsors also supported the development of multiple pathways to sponsor research, which is reflected in Figure 4. 3 While a primary pathway is for the SERC to "respond" to requests for proposals on topics of interest to various sponsors within the DoD and beyond, the OSD sponsor and the SERC developed an alternate pathway to "incubate" creative research ideas on topics of broad interest to the DoD.Accordingly, the sponsor established a summer incubator program which helped engage a wider set research faculty.The sponsor(s) would identify a broad set of research priority areas, and researchers could respond to with short proposals to explore ideas, primarily to fund graduate student research over the summer.The sponsor selected several research topics, each of which received a relatively small amount of seed funding.The results were presented in the fall with the potential for significant follow-on funding if the results proved interesting.The sponsor invited the DoD Services and Federal agencies to hear the results and consider independently funding further research.This effort grew both the network of researchers and potential funding agencies.Additionally, the sponsors developed a culture of being open to other "ideas" from the academic community-and when responsive to an evolving set of priorities and challenges, a number of these research ideas were sponsored and supported.Finally, the DoD has occasionally called upon the SERC to establish and "connect" panels of subject matter experts to a topic of urgent and immediate concern and interest-to rapidly develop potential options and recommendations.
licited proposals; and to engage focused panels.This work had led to enhanced teaming and increased participation across the SERC universities.The SERC also started collecting data on university participation, which has informed decisions to shape the university network over time.The sponsors' advisory board of senior executives representing various DoD agencies offered guidance to the SERC to ensure that research remained relevant to their needs.By conducting a short advisory board session in conjunction with the annual research review, and by holding deep dives in the summer into portions of the SERC research strategy, OSD developed a network of DoD organizations interested in the success of the SERC.Another key aspect of sponsor engagement has been the ability to work on highly focused, and sometimes classified, research projects that are very applied in nature.While universities have a high variance in their ability and desire to work on classified projects, this is an avenue that the SERC has pursued.On the one hand, it required the sponsors and the SERC leadership to work with the contracting officers to create classifications at the task or sub-task level-to allow broader participation by faculty without clearances; on the other hand, it required the SERC to develop a sense for which universities and faculty are most supportive of research at higher levels of classification.
been astounding.At the last center review, the SERC had over 85 faculty dedicated to systems engineering research engaged as Principal Investigators from the collaborating universities, 15 holding leadership positions in professional societies or journals, with hundreds of active doctoral students.Greater than 50% of all US-based MS-SE and PhD-SE graduates every year come from the SERC universities.SERC faculty and students were responsible for over 500 conference and journal publications in the last 5 years.The SERC's first project, the Systems Engineering Body of Knowledge (SEBoK) has become the independent global source of SE knowledge, with more than 30,000 unique visitors per month and over 5 million total page views.These impacts would not be achievable by any one university.It can take a significant amount of time and effort to conceptualize and then fully realize the benefits of a networked research center.Our experience with the SERC, over the course of the past 15 years, has demonstrated that the research results and impact on the field are worth this extra effort.The SERC network has been far more impactful than if Stevens had been the lone university in the UARC.The authors are gratified to see that the DoD has embraced this networked model for its recent research centers.The latest UARC, focused on Tactical Autonomy (RITA at Howard University in Washington DC) has been established with the SERC as its model-and many of us who have help the SERC in a leadership capability are formally and informally supporting RITA as well.