EVALUATION OF THE NUTEK-VINNOVA PROGRAMME IN COMPLEX TECHNICAL SYSTEMS 1997-2001

V I N N O V A R E P O R T V R 2 0 0 4 : 0 3 EVALUATION OF THE NUTEK-VINNOVA PROGRAMME IN COMPLEX TECHNICAL SYSTEMS 1997-2001 Utvärdering av ett FoU-program i Komplexa Tekniska System 1997-2001 E R I K A R N O L D E T A L, T E C H N O P O L I S L T D

Titel / Title: Evaluation of the NUTEK-VINNOVA Programme in Complex Technical Systems 1997-2001 - Utvärdering av ett FoU-program i Komplexa Tekniska System 1997-2001 Författare / Author: Erik Arnold et al, Technopolis Ltd Serie / Series: VINNOVA Rapport VR 2004:03 ISBN: 91-85084-05-0 ISSN: 1650-3104 Utgiven/ Published: April 2004 Utgivare / Publisher: VINNOVA - Verket för Innovatonssystem / Swedish Agency for Innovation Systems VINNOVA Diarienr / Case No: 2002-00241 REFERAT (syfte, metod, resultat): 1991 deklarerade den svenska regeringen i en rapport att kunskap i att hantera stora system saknas i svenska företag trots att kompetens i systemtänkande är mycket viktig för den svenska industrins konkurrenskraft. Både forskning och undervisning måste etableras. För att komma till rätta med denna brist föreslogs en större satsning på forskning och utbildning inom systemteknologi. Resultatet av ett antal seminarier arrangerade av NUTEK och IVA blev ett förslag till ett pilotprogram Systemteknologi och metodik för utveckling av komplexta tekniska system. Programmet startade 1994. Det var från allra första början en gemensam satsning mellan akademin och industrin där regeringen via NUTEK finansierade hälften av kostnaderna och industrin den andra hälften, huvudsakligen genom deltagande i projekten med egen arbetskraft. En första utvärdering betecknade programmet som klart framgångsrikt och rekommenderade en fortsättning i större skala efter dess planerade slut 1997. Programmet fortsattes därför med ökade resurser under namnet Komplexa Tekniska System och de första projekten startades 1998. Den aktuella utvärderingen initierades efter programmet slut 2001. Målet var att bedöma resultaten och effekten av programmet, speciellt från ett industriellt perspektiv, liksom att bedöma de akademiska impulserna till att öppna nya forskningsområden och nya kurser. Avsikten var också att fånga in synpunkter från de som deltagit i projekten. Utvärderarna ombads också att ge sina åsikter om framtida forskningsinriktningar och industriella insatser inom området i Sverige. För utvärderingen kontrakterades Dr. Erik Arnold från Technololis Ltd som projektledare. Utvärderingsrapporten kompletteras med en mer detaljerad redogörelse av bakgrunden till programmet, dess historia inklusive styrgruppens aktiviteter samt en statistiksummering av resultaten. ABSTRACT (sim, method, results): In 1991, a programme in the area of Systems Technology was suggested in a Swedish government report, which stated as a conclusion: Knowledge how to handle large systems is lacking in Swedish companies although competence in systems thinking is very important for Swedish industry competitiveness. Both research and education need to be established. Following a series of seminars, organised by the Swedish Agency for Technology Development (NUTEK) and The Royal Academy of Engineering Sciences (IVA), a decision was taken to initiate a pilot programme to be named Systems technology and methodology for Development of Complex Technical Systems. The programme started to operate in 1994. From its initiation, the programme was a cooperative effort between academia and industry, with government financing one half of the cost and industry the other half, primarily through own work. A first evaluation designated the programme as clearly successful and recommended a continuation of the programme on a larger scale after the end of term in 1997. This was in fact done and the first projects in the continued and enlarged programme Complex Technical Systems started in 1998. The present evaluation was initiated after the end of the programme 2001. The objective was to assess the outcomes and impact of the programme, especially from an industrial point of view, but also to address the academic achievements, opening up of new scientific fields, new curricula etc, and finally the overall views of those who had participated in the projects. The evaluators were asked, also, to give their views on future directions for research and industrial efforts in the area of Complex Technical Systems in Sweden. For this evaluation, Technopolis Ltd, was contracted, with dr Erik Arnold as team leader. The evaluation report is supplemented (in Swedish) by a more detailed account of the background and history of the programme, a summary of the activities of the steering group as well as some statistics on the programme. I VINNOVAs Verket för Innovationssystem publikationsserier redovisar bl a forskare, utredare och analytiker sina projekt. Publiceringen innebär inte att VINNOVA tar ställning till framförda åsikter, slutsatser och resultat. Undantag är publikationsserien VINNOVA Policy som återger VINNOVAs synpunkter och ställningstaganden. VINNOVAs publikationer finns att beställa, läsa eller ladda ner via www.vinnova.se. Tryckta utgåvor av VINNOVA Analys, Forum och Rapport säljs via Fritzes Offentliga Publikationer, www.fritzes.se, tel 08-690 91 90, fax 08-690 91 91 eller order.fritzes@liber.se VINNOVA Swedish Agency for Innovation Systems publications are published at www.vinnova.se

Evaluation of the NUTEK-VINNOVA programme in Complex Technical Systems 1997-2001 Utvärdering av ett FoU-program i Komplexa Tekniska System 1997-2001 Erik Arnold et al, Technopolis Ltd 1

Introduction The present evaluation report concludes NUTEK/VINNOVAs programme in the area of Complex Technical Systems. The programme itself was closed in August/September 2001 after being in operation for almost eight years. Already in 1991, a programme in the area of Systems Technology was suggested in a government report IT2000 (Dep of Industry Ds 1991:63) which stated as a conclusion: Knowledge how to handle large systems is lacking in Swedish companies although competence in systems thinking is very important for Swedish industry competitiveness. Both research and education need to be established. Following a series of seminars, organised by NUTEK and The Royal Academy of Engineering Sciences (IVA), on Complex Systems in 1992, a decision was taken to initiate a pilot programme to be named Systems technology and methodology for Development of Complex Technical Systems. The programme started to operate in 1994. From its initiation, the programme was a cooperative effort between academia and industry, with NUTEK/VINNOVA financing one half of the cost and industry the other half, primarily through own work. A first evaluation was concluded in 1996. It designated the programme as clearly successful and recommended a continuation of the programme on a larger scale after the end of term in 1997. The first call for proposals within this new/continued programme was made public in the autumn 1997 and the first projects started in 1998. A second evaluation was concluded in May 2000, where 44 projects were evaluated by an international panel of 10 peers. This second evaluation primarily aimed at assessing the scientific quality of the projects. It was based on written submissions and individual presentations to the panel by the project leaders and, in some cases, also by industrial partners. The present evaluation, the third one, was initiated after the end of the programme. The objective was to assess the outcomes and impact of the programme, especially from an industrial point of view, but also to address the academic achievements, opening up of new scientific fields, new curricula etc, and finally the overall views of those who had participated in the projects. The evaluators were asked, also, to give their views on future directions for research and industrial efforts in the area of Complex Technical Systems in Sweden. For this evaluation, Technopolis Ltd, was contracted, with dr Erik Arnold as team leader. The evaluation report is supplemented (in Swedish) by a more detailed account of the background and history of the programme, a summary of the activities of the

The Programme Board and VINNOVA staff lined up after their last meeting in 2001. From left to right. First row: Bengt Larsson - VINNOVA, Lena Mårtensson - KTH, Anders Hedin & Barbro Atlestam - VINNOVA. Second row: Jonas Bjarne & John Graffman - VINNOVA, Ove Borgström - Ericsson, Lars-Göran Rosengren - VOLVO TU, Arne Otteblad, Ingemar Karlsson - FMV, Bengt Sjögren - Kompetens Lagman AB, Ulf Holmgren - VINNOVA. Absent were: Hans Skoog - ABB, Gustaf Olsson - Lunds Universitet, Karl-Einar Sjödin & Ingvar Åkerstedt - VINNOVA.

Contents The Complex Technical Systems Programme at NUTEK and VINNOVA: An Evaluation by Technopolis Ltd, June 2003 Appendix 1/Bilaga 1: Om Komplexa tekniska system - programmens bakgrund, syften och genomförande (in Swedish) Appendix 2/Bilaga 2: Styrgruppens metoder för verksamhetsstyrning och intervenering (in Swedish) Appendix 3/Bilaga 3: Statistics from the programme

The Complex Technical Systems Programme at NUTEK and VINNOVA: An Evaluation Report to VINNOVA Erik Arnold John Bessant Enrico Deiaco James Stroyan Ben Thuriaux Shaun Whitehouse Rapela Zaman June 2003

Contents Summary 1 Introduction and Methods 1 1.1 What are Complex Technical Systems? 1 1.2 Scope of this Evaluation 2 1.3 Methods and Limitations 2 1.4 Acknowledgements 3 2 The Complex Technical Systems Programme (KTS) 5 2.1 The First KTS Programme, 1993-1996 5 2.2 Transition 7 2.3 The Second KTS Programme, 1997 1999/2000 7 2.4 Evaluation of Projects in 2000 11 3 What the Programme Did 13 3.1 Programme Composition 13 3.2 What the Participants Did 22 4 Results of the Programme 29 4.1 Questionnaire Analysis 29 4.2 Results Reported by Projects 40 5 Management and Administration 41 5.1 The Programme Board 41 5.2 Interviews with Board Members 42 5.3 NUTEK Support and Procedures Questionnaire and Interview Evidence 43 5.4 Programme Administration An Evaluation Perspective 45 6 Project Experience and Case Studies 46 6.1 Project Participant Views 46 6.2 Case Studies 49 7 Complex Technical Systems in an Agency for Innovation Systems 63 7.1 Knowledge Production and Innovation 64 7.2 The Theoretical Problem 67 7.3 The Human and Social Dimension of Technology 68 7.4 Interdisciplinarity 71 7.5 Competitiveness and Innovation Systems 72 8 Conclusions and Recommendations 76 8.1 Conclusions on KTS as an Advanced Technology Programme 76 8.2 Implications for VINNOVA as an Innovation Systems Agency 78 8.3 Next Steps 84 Appendix A Questionnaires to Project Participants 88 Appendix B Questionnaire Non-Response Analysis 103 B.1 Additional Questionnaire Analyses 105 i

Summary This evaluation report to VINNOVA tackles two somewhat different questions. First, we evaluate the Complex Technical Systems (KTS) programme. We find it to be a strong programme, well tuned to industrial needs and producing many useful outputs, including both re-usable knowledge and results of more direct use in industry. Second, we consider what VINNOVA can learn on a more general level from the programme for its own role in the innovation system and about how to run significant technology and innovation activities. Technical complexity and what we might call systemicity are important themes in modern innovation systems. This is partly because we face more technological opportunities than previous societies and partly because our society operates at unprecedented scale. Mastering complex technical systems, and especially their design is therefore an important agenda. The first KTS programme ran originally from 1993 1996. This evaluation covers the second KTS programme, which ran after a gap of a year from 1997 to 2000. The issue of complex technical systems was much discussed at the start of the 1990s, when it was generally recognised that this was an economically important area. NUTEK launched an R&D programme in 1993 to find out how better to design such systems, and thereby to support the competitiveness of Swedish industry. A larger successor programme was launched in 1997, which is the subject of this evaluation. This aimed to support Swedish competitiveness by reinforcing knowledge about systems development and devising general methods that could be used to tackle a range of types of complexity. The programme was later cut back by 4 months owing to budgetary restrictions. A key strategic change during the life of the programme was the introduction of horizontal projects, aiming to generalise and transfer knowledge between the other, more domain-oriented sub-programmes. The KTS programme generated well over 265 MSEK of activity across 130 organisations. Half the effort went on the broad field of systems development. Most of the effort went on this and other specific complex technical domains. The smallest amount went to horizontal projects, aiming to synthesise and develop more generic knowledge an aspect which industry was not inclined to co-finance. Most of the projects in the programme were rather small and long, with 18 (a quarter) having a budget above 5 MSEK across a three year period. Project leaders were predominantly university personnel, who were positioned as potentially key nodes in the wider national network of the programme. Projects were strategically important and close to the core business of those who performed them. They were often incremental, reflecting the strong industrial influence in project definition. While this influence appears to have been more important than is usual, in many respects the KTS projects had similar characteristics to those carried out in other advanced technology programmes, with knowledge outputs being central. These tend to produce intermediate products, useful in doing further development and research, rather than final products or processes. ii

Our questionnaire analysis suggests that participants did well in satisfying their own goals through the programme, but is less clearly positive about the projects effectiveness in reaching programme goals. Most of the projects were additional and there appears to have been little free riding in the programme. The main outputs were knowledge and networking. Despite the long term nature of the projects, some short term commercial exploitation had already been achieved. New knowledge was generated but externalities from the projects, in the form of knowledge and capability transfers, had been limited. Both industrial and academic participants felt that the benefits of being involved in the KTS programme had very significantly outweighed the costs. The overall picture, then, is consistent with the impression we have already formed of projects that have strong industrial steering but a weaker link to wider social and technical goals. NUTEK had well-established routines for programme definition and management, which in general performed well. These were documented in a benchmarking report 1 commissioned by NUTEK in the mid-1990s, and have subsequently been compared with other R&D programme management practices, especially in the Nordic area. 2 We chose, therefore, in this evaluation not to attempt an exhaustive review of the programme management processes involved in KTS. Instead, we used our questionnaire, interviews and discussions with programme management to look for signs of trouble, and to get an overall assessment of programme management from the user perspective. The programme board comprised experienced and influential stakeholders and provided helpful continuity with the past, a powerful knowledge base and a resource for mobilising projects. It took measures that resulted in high industrial influence over the programme and its projects. However, the great size of this second KTS programme meant that both the board and the administration at NUTEK/VINNOVA were stretched. The board stressed programme conferences as a way to bring the programme together. This was partly successful, though the wide scope of the programme proved to be a barrier to bringing together people with very different interests. The rather bottom-up way the programme had been populated with projects led to fragmentation. Generic complexity questions did not arise spontaneously from industry or from academics. Participants were generally very positive about the quality and timeliness of administration. From our evaluation perspective, and indeed in terms of programme management s need for basic management information and some kind of statistical process control, the programme did not have adequate Information Systems support. Discussions with project participants confirmed that many of them worked in rather long-standing networks with industry, though new relationships were also being formed. University people were key in establishing many projects, while end users were rarely involved in project definition or execution. Projects generally claimed to be developing tools and methods for complex technical systems development and 1 2 Erik Arnold and Paul Simmonds, Programme Management Benchmark Study; Final Report to NUTEK, Brighton: Technopolis 1997 James Stroyan and Erik Arnold, Comparative Study on Administrative Burdens and Rules fo Procedure between the EU Research Programmes and those of the Individual Member States, project IV/98/06, Luxembourg: European Parliament, 1998 iii

many to be raising systems design capabilities in universities and industry, including through postgraduate education. Project leaders found the KTS programme rather broad and, in some cases, hard to identify with. NUTEK administration got high marks from the project leaders. The case studies showed that there is a wide range of project types. Some projects had succeeded in integrating human factors, others had not. Those that worked well had produced useful knowledge and applicable results in the form of inputs to development and changed working practices. Horizontal project cases aiming to synthesise domain-specific work in the programme in order to develop generic systems understanding suffered from the lack of generalisable results and tended to become (useful) state of the art reviews rather than true syntheses. KTS tried, as the test pilots say. to push the envelope : that is, to see how far it could go beyond what is normal in this kind of technology programme. At the very least, we can conclude that KTS tackled some of the more difficult aspects of modern technological development. These difficult aspects include the desire to integrate human beings and human factors into thinking about complex technical systems and the programmes explicit aim to be interdisciplinary. An overall conclusion of this evaluation is that the KTS programme was interesting, useful and well executed. It was additional, it raised capabilities, developed new knowledge and techniques and led to important benefits for participants. Our evaluation of the programme in its context and in its time is positive. However, we do evaluation not only in order to make judgements about whether programmes reach useful results or whether they are well run. Evaluation is an important way to learn, and KTS offers some interesting and, we think, useful lessons. The be useful, we need to consider these lessons using current theory, since VINNOVA as an agency has been conceptualised on the basis of innovation theory that was only just beginning to be formed in the early 1990s, when the KTS programme was conceived. Also, in trying to make practical as well as intellectual progress in tackling generic complexity issues in industrial design practice, KTS bit off more than it could chew. This is not a normal type of conclusion in an evaluation of NUTEK or VINNOVA programmes, but neither is KTS a normal programme. Like other NUTEK/VINNOVA programmes, KTS is a Mode 2 programme, involving industry, university and other R&D workers, as opposed to a Mode 1 disciplinary activity. This type of research is becoming increasingly important as new technologies evolve at the boundaries between disciplines and as the capacity to do R&D increases in all parts of the economy. In some cases, simultaneous R&D and more fundamental work may be helpful in supporting and creating opportunities for innovation. Researchers may also move back and forth between the two Modes, according to need. Knowledge needs are constantly changing. Programmes can help create communities of Mode 2 practice, in order to spread generic knowledge and also to help achieve links with longer term work that can break bottlenecks in the innovation process, such as those encountered in KTS. A key bottleneck was the lack of an adequate generic systems theory, or a hypotheses within the KTS programme about what such a theory might look like. The stress laid iv

in the programme on industrial participation tended to squeeze out the more fundamental work needed, and the KTS programme lacked incentives and instruments that could overcome this market failure. Different project structures, for example involving more end users, could help improve the ability of the research funded to integrate human factors. Other configurations (such as larger, multidepartment projects) might have been useful for increasing the amount of interdisciplinary work in the programme. Not least, the very technical focus of the programme has tended to push these aspects down the agenda. In an innovation systems agency context, it would make sense to take account of the different ways that low volume complex product systems and high-volume complex products are produced, because these appear to require different hard and soft technological approaches. It seems that VINNOVA has inherited an inadequate range of funding instruments from its predecessors. The problems KTS tried to tackle could not be solved without a combination of instruments to involve industrial co-operation but also to fund more strategic or fundamental research, for which industry simply will not pay because of the well known market failures associated with longer term research. The structure of the Swedish knowledge infrastructure continues to be an impediment to tackling many of the more important challenges in knowledge production and use, such as interdisciplinarity, scale and the shift in the locus of knowledge production to being primarily outside the universities. There is a continuing need for a change agency here. The governance and incentive systems used need to be tuned to the activities of individual programmes. Those used with KTS tended to lock the programme into the shorter-term concerns of industry and to lock out the work that should have been present to meet its longer term needs. VINNOVA needs to counterbalance the excellent tradition of stakeholder involvement in programme design that it has inherited from NUTEK with both economic and technological analysis capabilities, in order to balance short- and longer-term needs. VINNOVA is grappling with the question: What, if anything, should follow on from the KTS programme? It would be possible to focus a new programme design by soliciting these kinds of longer-term business needs driven questions and confronting them with a set of more research-driven ones. The additional element required is analysis. Before trying to launch a programme, it is necessary to Understand the state of the art, to make sure that the answers to the programme s research questions are not already available or that the proposals do not introduce some new mission impossible to the agenda Scope the potential areas of social and economic impact. This is not to say that an economic impact or cost-benefit analysis should be attempted, for this is methodologically untenable. Rather, it is important simply to confirm that the intended action is likely to affect large or critical parts of the Swedish innovation system rather than small or unimportant ones It would make sense to complement these analyses with an explicit programme activity that maintains a technology watch on foreign developments. In the short term, VINNOVA would have at least three options in terms of the type of programme to put in place v

1 A combined co-funded industrial programme and a 100%-funded more researchintensive strand, somewhat in the style of TEKES 2 A call for tenders to establish two or three new competence centres or, perhaps, a single, much larger multi-site competence centre 3 An invitation to construct an industry consortium or consortia, working with long term research questions in the area of complex technical systems. The MediaLab at MIT could provide a model, where companies buy into research programmes for extended periods. IMEC, the major microelectronics research institute in Flanders, works in a similar way The main criteria for choosing among these forms is the extent to which the problems identified are likely to persist, the extent to which tackling them involves cutting across existing departmental lines in the universities and the willingness of industry to become involved as a long-term partner. In the longer term, VINNOVA has an opportunity to use the complexity agenda as a platform for negotiating a pattern of working together with research councils and foundations that goes beyond today s rather minimal level of co-ordination. vi

1 Introduction and Methods 1.1 What are Complex Technical Systems? Technical complexity and what we might call systemicity are important themes in modern innovation systems. This is partly because we face more technological opportunities than previous societies and partly because our society operates at unprecedented scale. The two things are strongly interconnected. Equally, since electronics matured into microelectronics and gave us the power to work with millions of bits of information and processing elements at a time, information technology has opened the door to huge possibilities. Today, the average Swedish household probably has more information processing power than did most countries a few decades ago. The technology lets us address more complex situations than before and it allows us to invent more complex situations to address. Complexity in this simple sense of having many parts is not necessarily hard to handle. Imagine driving from Borås to Gothenburg. For simplicity, let s do it after meeting our grandparents in church on Christmas Eve, when there is almost no traffic. We navigate the streets of Borås, take the motorway past Landvetter Airport and soon arrive in Gothenburg, where we navigate to our destination. If we do not know the way we may use a map. But, despite the fact that we are using a small corner of an immense continental road network, the problem is not much more difficult than the one our grandparents face in walking home from church to their house in Borås or would have faced, doing the same 50 years ago. Shift the perspective now to that of the Swedish National Roads Administration (Vägverket) and the local councils who between them design, build and manage these same roads. They have to consider not one car but many, and crucially the interactions among them. What happens when the shift change at Torslanda meets the cross-town morning traffic? When the morning commuters from Borås and the newly-landed Stockholmers in their taxis from Landvetter all converge on the motorway exit, where the traffic has slowed to a crawl because of the sheer number of vehicles interacting? To answer these questions, we first have to understand the roads not just as a network but as a system, whose parts interact. Because of these interactions, properties emerge that are not obvious from looking at the network. The planners find they need complex computer models with feedback loops to account for these emergent properties such as the traffic jams. They fit the road network with sensors, link the sensors to computers and the computers to the traffic lights. They provide real-time traffic information to the car drivers to influence their behaviour. They struggle to manage the highway engineers, electronics specialists, social and behavioural scientists, economists and mathematical modellers they bring in, but who hardly understand each other. They argue about whether the SNRA should modify the highways or the local council should change the layout of the one-way system. Just when things seem to be getting better, a tram breaks down at a central junction. Drivers take to the back streets to avoid it, and unexpectedly bring most of the 1

downtown area into gridlock, showing how vulnerable the new street layout is to disruption. These elements big networks, complex interactions with feedback loops and emergent properties, interdisciplinarity, the influence of human behaviour recur with increasing frequency in our societies, which is why complex technical systems are so important. A significant part of OECD industry, and especially that of Sweden, builds operates and makes components for complex technical systems, so knowledge and skills about these systems are economically very important. The cars in the Gothenburg traffic are themselves complex technical systems, and the production systems that make them even more so. The problems when complex technical systems go wrong are economically significant, too whether we think of the time wasted by all the people sitting in the traffic jam looking at Liseberg go past in slow motion, or the bill the tax payers have to foot when the latest big administrative systems development project goes five times over budget and still does not work. 1.2 Scope of this Evaluation Mastering complex technical systems, and especially their design since most problems are designed in from the start is therefore an important agenda, not only for Sweden but also in other countries (one reason why we think this evaluation will have international interest). The importance of complex technical systems was recognised in Swedish policy debate already in the early 1990s. NUTEK (the Swedish National Board for Industrial and Technological Development) launched an R&D support programme to tackle the question. This evaluation covers the second Complex Technical Systems (KTS) programme run by NUTEK. The first ran originally from 1993 1996, though some projects lived on beyond the formal end of the programme, and several were incorporated (sometimes in modified form) into the second programme, which ran after a gap of a year from 1997 to 2000. 1.3 Methods and Limitations Evaluation of this type of activity is by no means an exact science. The methods at our disposal are individually not completely reliable and are often subject to bias. To reach conclusions in which we can believe, therefore, it is important to combine different ways of looking at the entity to be evaluated. We can have confidence in our findings and conclusions to the extent that the indications from these different tools and techniques converge. To tackle the KTS programme, we Reviewed programme documentation Reviewed other relevant literature on complexity Interviewed programme management Interviewed six of the eight members of the programme Board Sent a questionnaire to 183 participants in the programme, asking about their motives and goals in participating, their achievements and their experience of participating Interviewed a selection of 21 project participants to deepen our understanding about participation 2

Participated in four meetings with VINNOVA personnel and other interested parties to discuss our methods and the emerging conclusions of the evaluation We also conducted a number of illustrative case studies, intended to help both the reader and ourselves understand in a more qualitative way how the programme operated. While we naturally try to be as rigorous as we can, an evaluation such as this finally represents a number of judgements. We have tried to make clear distinctions between what we regard as data and our judgements, since the reader may want to interpret the data differently from us. The evaluation team has collectively conducted upwards of one hundred R&D programme evaluations, so our judgements are clearly informed by experience, but also by research and theory about research, innovation and innovation systems. Finally, however, this kind of report can never be Truth. We see it rather as one contribution among others to a debate about research and innovation policy, and hope that it is a valuable input. As concerns the questionnaire, in the middle of August 2002, a total of 183 questionnaires were e-mailed to participants in the KTS programme. Almost half (43%) of the questionnaires have been completed, though this figure rises to 49% if we discount from the population those recipients who were either not able to be contacted or who indicated that their involvement / recollection was insufficient to permit a detailed response. We made efforts to contact all non-respondents, both by email and by telephone. Exhibit 1 sets out the responses obtained by 18 th October 2002, at which point we froze 3 the analysis. Exhibit 1 Questionnaire Responses Research base Industry Total Share of total Completed 41 37 78 43% Undeliverable 1 5 6 3% Refused 3 22 25 14% Not responded 26 48 74 40% Total (mailed) 71 112 183 100% As the Exhibit indicates, the responses rate is much higher from academics (58%) than from industry (33%). However, the responses of the two populations are surprisingly similar. (Where they differ in an important way, we show separately the responses from the two populations.) We explored the responding and nonresponding populations to see whether there were other important differences. Two potential sources of bias emerge. One is that non-respondents were more likely to come from the Systems Development sub-programme. The other is that people from single-organisation projects were somewhat less likely than others to respond. 1.4 Acknowledgements The evaluation team is grateful to the large number of people who helped us in this work: interviewees; the many people who filled in our questionnaires; and not least 3 The 2 questionnaires since received have been disregarded 3

the steering committee, programme management and evaluation team at VINNOVA, the agency which inherited the KTS programme and a great deal more from NUTEK. The responsibility for mistakes, misunderstanding or misinterpretations, of course, lies wholly with the ourselves. 4

2 The Complex Technical Systems Programme (KTS) The issue of complex technical systems was much discussed in Swedish R&D policy circles at the start of the 1990s, when it was generally recognised that this was an economically important area. NUTEK launched an R&D programme in 1993 to find out how better to design such systems, and thereby to support the competitiveness of Swedish industry. Following a positive evaluation and a gap of a year during which there we negotiations about financing, a larger successor programme was launched in 1997, which is the subject of this evaluation. The programme aimed to support Swedish competitiveness by reinforcing knowledge about systems development, devising general methods that could be used to tackle a range of types of complexity. The programme was executed, but cut back by 4 months owing to budgetary restrictions. A key strategic change during the life of the programme was the introduction of horizontal projects, aiming to generalise and transfer knowledge about complex systems design between the other, more applications-oriented subprogrammes. 2.1 The First KTS Programme, 1993-1996 Technical complexity appears to have been much debated in Swedish R&D policy circles at the start of the 1990s. One symptom of this was evident in the IT2000 report, which was produced by a national committee in 1991, considering the industrial and research policy implications of information technology (IT). The committee observed that the capability to deal with large systems is in short supply in all large firms, while it is exactly this systems capability that is Swedish companies most important competitive resource. The idea of a complex technical systems programme was considered inside NUTEK and in the short-lived Swedish Research Council for Engineering Sciences (TFR), both of which had been set up in 1991. The idea of an R&D programme on complex technical systems found support both in NUTEK s routine researchers surveys and in a joint seminar between NUTEK and the Royal Swedish Academy of Engineering (IVA) in 1992. A group convened by TFR suggested in 1992 that there should be an R&D programme on the subject. This did not find favour with the council, but the following year after a considerable amount of preparation NUTEK decided to run a KTS programme, which was actually entitled Systems technology and development methodologies for complex technical systems. A paper 4 from a joint IEEE and IFAC (Institute of Electrical and Electronic Engineering and International Federation for Automatic Control) summarises a number of challenges, which were in the air at the time Growing combinatorial complexity in engineering systems being designed. Systems are made up of an increasing number of sub-systems and components. As the number of system elements increases, the number of interactions among elements rises even faster 4 A Beneviste, K J Åström, P E Caines, G Cohen and L Ljung, Facing the challenge of computer science in the industrial applications of control, October 10, 1991 5

The growing need for interdisciplinary approaches, as systems bring together different technologies (such as electrical, mechanical, hydraulic, and so on) Barriers to the movement of knowledge among engineering disciplines. Most important, the growing use of digital circuitry in control not being accompanied by a take-up of mathematically based software engineering techniques and modularity, used in computer science to manage complexity and assure quality. The key barrier here was seen as a compartmentalised education system Insufficient academic/industrial interaction on these problems, leading to insufficiently global understanding of the issues in industry and a research tradition insufficiently grounded in experience NUTEK launched its first KTS programme, based on a plan 5 devised with the assistance of a group of industrial and research institute people. These extremely competent stakeholders provided both their best assessment of the situation and a crucial link between the KTS programme and the Swedish communities who should be involved. However, there was no more formal state of the art or literature review in what was to prove a theoretical minefield. The programme s goals were to find general methods, tools, descriptive models etc to develop, manage, produce and use complex technical systems and master the negative aspects associated with complexity. It was to involve industry, research institutes and universities, and was to focus on problems of relevance to Swedish industry. Implicitly, it was understood that an important proportion of Swedish industrial activity is in the production of such systems. Complexity was seen as positive in that it provided a means to increase systems performance. However, the programme plan also viewed complexity as having a number of negative aspects many of which are cognitive, rather than being in the nature of the technologies themselves Non-deterministic (or poorly understood) relations among system components and sub-systems The need to integrate across multiple systems and technologies (and therefore disciplines) Inability to communicate among disciplines, causing difficulties in understanding individual complex systems Combinatorial complexity Complexity in the sense of the complicated or difficult was explicitly excluded from the programme. Thus, for example, complex flows were out of scope. Rather, the programme focused on complexity in relation to systems. The programme s strategy was to start with problem areas or applications areas, where complex technical systems are produced, and to develop projects, which would tackle problems specific to these. One of the criteria for project selection was intended to be the use of scientifically based methods that could be used in several 5 NUTEK, with the co-operation of Ove Borgström Ericsson Telecom, Ingemar Carlsson FMV, Sven Gunnar Edlund STFI, Krister Gerdin SMHI, Ulf Rehme Saab Scania, Lars Göran Rosengren Volvo, Programplan för systemteknik och utvecklingsmetodik för komplexa tekniska system, NUTEK, 1993-04-19 6

applications areas. (The programme was not alone in thinking in this way about generalising from the particular. The UK s Engineering design Initiative launched a few years earlier similarly sought to generalise from particular areas of design to establish a science of design. 6 ) Projects needed to be clustered, in order to achieve more than trivial improvements and so as to contribute to the development of more generic methods. In the 1993-6 programme, there were three such sub-programmes 1 Design and development methods for technical systems products 2 Technologies for design, operation and maintenance of complex technical systems 3 Structuring distribution systems and networks The mid-term evaluators expressed 7 satisfaction with the way the programme had progressed, and recommended its continuation on a larger scale. 2.2 Transition The first KTS programme was originally intended to run from July 1993 to June 1996. It was extended by six months because the state decided to change NUTEK s financial year-end from June to December. However, during 1997, NUTEK was forced to enter negotiations with the new research foundations that had been set up by the conservative government using the Wage Earner Funds of the late 1980s. With the social democrats back in power, the new government was determined to take a high degree of operational control of these independent foundations. In the event, the foundations proved more difficult to influence than the government had anticipated and the negotiations dragged out for much of the year. The uncertainty prevented the KTS programme from buying new projects but it renewed contracts with existing ones as these fell due, in order to safeguard the teams involved. The outcome of the negotiations was not the effective take-over of the foundations, for which the government had hoped, but a modus vivendi where the Strategic Research Foundation in particular took over a number of NUTEK funding activities. As a result, NUTEK s Teknik division, the part of the organisation that funded R&D, lost about a third of its budget and the same proportion of its staff (a number of whom transferred to the Strategic Foundation). The effects of this negotiation for KTS were twofold. First, it meant that almost a year was lost at the beginning of the three-year period. As a result, the programme effectively ran through much of 2000, with parts running over into 2001. Second, the loss of staff put the administration of the programme under pressure, since for this programme the budget remained unchanged while the manning was reduced by a third. 2.3 The Second KTS Programme, 1997 1999/2000 KTS was continued with a much larger scope and budget in 1997-1999. Confusingly, the KTS programme was clustered with a number of other existing activities (including the Telecommunications programme) to form a complex technical systems 6 7 TFR, Systemteknik en studie utförd för TFR, Stockholm: TFR, 1992 Ib Grønvaldt och Jouko Soukas, Industrirelevansen i programmet Systemteknik och utvecklingsmetodik för komplexa tekniska system: Utvärdering, Stockholm: NUTEK, 1996:61 7

resultatområde results area within NUTEK. These other activities were technically related to KTS mainly as tools and applications, but were not in themselves focused upon complexity issues. NUTEK s annual activity plan for 1997 explains the then perceived direction of the KTS programme. The growing complexity of technical systems is often connected to the interplay of three factors Growing and increasingly narrow disciplinary knowledge Improved IT methods Increasing performance requirements for products and systems The programme plan argued that the need to master complexity was rising. Complexity itself was increasing as a result of Increasing numbers of sub-systems built on different technologies mechanical engineering, aerodynamics, hydraulics, pneumatics, electronics and other technologies are being integrated into systems and must be made to work together Increased security and quality requirements Environmental and life-cycle thinking mean that a growing number of factors and parameters must be tackled during development and use of systems Cost minimisation and short lead times require increased organisation for product development and c-ordination among production units 8 The programme was justified by this rising importance of complexity for Swedish industry. The programme plan pointed out that the Swedish industrial structure is particularly strong in systems industries, and that Swedish products increasingly have a systems character. Swedish companies and organisations are also rapid adopters of IT. This meant not only that complex technical systems were important for Sweden but that increased capabilities in handling complexity should translate into increased competitiveness. Complex system capabilities were also required by SMEs, if they were to remain competitive as suppliers of components and sub-systems. Interdisciplinary R&D was seen as the required approach. The second KTS programme therefore had long-term goals of Securing and reinforcing knowledge about systems development, in order to preserve and increase the competitiveness of a key part of Swedish industry Developing general methods and tools which can be used in various applications Its short-term goals were to Link capabilities in cross-disciplinary networks and transfer knowledge between technology areas through increased cross-disciplinary co-operation among actors in research, education, industry and end-users 8 Komplexa Tekniska System, Programförklaring 1997-1999 (01) 8

Test and evaluate techniques and methods, and disseminate information about these through demonstrations and applications Specifically, the programme was expected to generate a broader and deeper knowledge about systems building to more actors. As a spin-off, it was hoped that it would also produce some immediately-applicable results through new methods and tools, which could be marketed as products in their own right. According to the programme s activity plan 9 Previous experience shows clearly that in complex technical systems it is not possible to start with a general approach and move from this to develop useful methods. In stead, it is necessary to begin by developing one or more examples of applications and then generalise methods and methodologies to other applications areas. To identify common problems among these areas is an important task for the programme. The programme was to be implemented through relatively large projects, each involving a small network of organisations spanning industry, public and private organisations, institutes and universities 10. However, NUTEK would finance only the participation of universities, research institutes and SMEs. Project results should both provide immediate benefits to participants and contribute to general understanding about handling technical complexity. The KTS results area had a total of six sub-programmes Systems Development handles rather basic questions about how to build complex technical systems in software. It emphasises systems technology, the integrity and security of IT systems, how to handle integrated product data and deals with software and hardware engineering methods Systems Architecture tackles the design and implementation of embedded systems, real-time systems and the architecture of software systems Telecommunications systems is part of the complex technical systems results area, but not of the KTS programme. It is a separate programme with a separate programme board and budget and limited linkage to the complex systems agenda of developing general approaches to complexity. It is not considered here but has separately been evaluated 11. Decision-Support Systems for Complex Operations focuses on the use of complex systems to support management and control in areas such as management of business processes, command and control and process control. Later in the programme, the problem of sensor-data fusion was added to this sub-programme Modelling and Simulation focuses on the development phase to produce tools and control systems Optimisation of Infrastructures is concerned with how to manage large technical systems such as the telecommunications and water supply and distribution networks 9 10 11 Resultatområdet Komplexa tekniska system, Verksamhetsplan för perioden 1997-1999 Universities in this report should be read as universitet och högskolor Erik Bruun, Donald Cox, Claude Guegen, Paul J Kühn and Peter Noll, Telecommunication: Evaluation of the Research Programme 93-96, Stockholm: NUTEK, R1997:5 9

An additional group of horizontal projects has been added, aiming to generalise results across applications areas and transport lessons between applications areas. The programme board assessed project proposals, aided by expert input. In addition to specific technical criteria related to the various sub-programmes, the board decided to consider the adequacy of proposals based on the following criteria 1 Problems should relate to complex technical systems with which Swedish companies or other organisations are working 2 The project idea itself 3 Quality in scientific and engineering terms 4 Active and continuing co-operation between the participating universities and institutes, on the one hand, and industry on the other 5 The probability that results would be adopted by industry 6 Project management 7 Interdisciplinary co-operation 8 The role of human factors in sub-systems or people as actors within the system 9 The development of scientifically based methods that can be used within several application areas 10 The mobility of researchers between the university/institute sector and industry 11 The participation of SMEs 12 Effects of the projects on the physical environment There appears at the start to have been a concern that KTS projects would be overly academic, as measures were put in place that were intended to increase the likelihood of securing industrial relevance. It seems likely that this worry about relevance led to reduced focus on some of the other criteria, notably that concerned with developing results of use within several application areas. The administration of the programme laid emphasis on the need for letters of intent to be signed by the companies involved, and for eventual contracts governing company participation to be signed at high level within the firms. The plan stressed the needs for companies to operate as project leaders in the majority of cases, but also said that it is important for the project leader to have a university education, and preferably a doctorate. It is not clear whether this principle was implemented, but it is notable that about three quarters of the project leaders were university employees. The size of the KTS programme meant that it could not be managed by one or two project officers only, as was often the case with NUTEK programmes, but was handled by a larger group. Broadly, this comprised those who had looked after the first KTS programme, plus a part-time involvement by project officers who had handled programmes that were integrated into the KTS results area in NUTEK. The project officers operated in a matrix across different parts of the NUTEK organisation. Members of the programme board and individual project officers were paired to up focus on individual sub-programme areas and clusters. In 1998, the programme board reflected 12 on the new projects it had approved and set a number of technical priorities for the individual sub-programmes. At the overall 12 Komplexa tekniska system 1997 1999, Programförklaring inför Andra ansökningsomgången 1998 10

level of the KTS programme, it decided that it should try to attract more activity from the process industries and that projects which handle human factors in complex technical systems should be included in all sub-programmes. The board did not, in its annual programme review, take up the question of generalising results from individual applications projects. It did, however, add an additional sub-programme Optimisation of Infrastructures which appears in the programme s activity plan for 1999. This sub-programme adds Large Technical Systems into the scope of the programme. It also added sensor-data fusion into the decision support subprogramme. The board made a useful innovation in monitoring, by proposing that a short-form project progress report be introduced, based on ABB s internal monitoring document. In 1999, there was a period of budget uncertainty, as the government reduced the available money for 2000 but left unclear whether any savings made in 1999 could be transferred to 2000. In the end this was permitted. However, the KTS budget for 2000 was still substantially short of the sum needed to bring all the ongoing projects to their planned conclusion. When the annual projects contracts were signed in March 2000, most were therefore told that the budget for their last four months of operation would not be available. One of the surprises of this evaluation is that these cutbacks had only a limited effect on projects, presumably reflecting the fact that they were primarily producing knowledge outputs of use in R&D rather than more specifically defined products and processes. However, at the strategic level, the cuts did more significant damage. In mid-programme, the board had become aware that the ambition of migrating lessons about complexity between different projects was not being well realised. It began looking for horizontal projects, which could generalise lessons from individual applications projects and translate these into general lessons on designing and managing complexity. It turned out to be difficult to launch such projects. While it was eventually able to run a handful of small-scale projects of this kind, its freedom to deliver the programme s intended general contributions to the understanding of complexity was clearly limited by the budget changes. Relevance and quality evaluations of the two KTS programmes have so far been positive, based largely on project by project consideration of what thy have done. Two concerns have been voiced about both programmes both in evaluations and at the level of the programme board. The first is about the effectiveness of this generalisation. To what extent does the programme add value to the projects in this way? The second is about the lower than desired level of involvement by the process industries, though there has been some success in increasing the number of projects relevant to the process industries during the latter part of the programme. 2.4 Evaluation of Projects in 2000 In May 2000, a total of 10 foreign evaluators were invited to review different parts of the programme. They saw a total of 44 projects, arranged in 5 blocks. The evaluation method used meant that no expert was able to develop an overview of the programme as a whole and that there is no overall assessment of programme quality. The detailed reports of the experts indicated that a good many of the projects had high scientific 11

quality and productivity, but also that there was a meaningful number of projects whose quality was more questionable. In some cases, the evaluators raised questions about the generalisability of project results, both in order to learn lessons about complexity in general and to other companies. In some of the sub-programmes, the evaluators sought more focus. But in general, they were strongly in favour of the programme and its continuation. The main importance for us, however, of the evaluation of projects in 2000 is that it confirms that the scientific quality of the KTS programme is in broad terms good and certainly acceptable. 12

3 What the Programme Did The KTS programme generated well over 265 MSEK of activity across 130 organisations. Half the effort went on the broad field of systems development. Most of the effort went on this and other specific complex technical domains. The smallest amount went to horizontal projects, aiming to synthesise and develop more generic knowledge an aspect which industry was not inclined to co-finance. Most of the projects in the programme were rather small and long, with eighteen having a budget above 5 MSEK across a three year period. Project leaders were predominantly university personnel, who were positioned as potentially key nodes in the wider national network of the programme. Projects were strategically important and close to the core business of those who performed them. They were often incremental, reflecting the strong industrial influence in project definition. While this influence appears to have been more important than is usual, in many respects the KTS projects had similar characteristics to those carried out in other advanced technology programmes, with knowledge outputs being central. These tend to be intermediate products, useful in doing further development and research, rather than final products or processes. 3.1 Programme Composition The findings presented here are based on data provided by VINNOVA. In some cases, data were incomplete, particularly data related to participant type (such as large or small firms). The data provided were therefore supplemented by information from the Complex Technical Systems Project catalogue, published by NUTEK in 1999. In addition, information on the size of firms was supplemented by Technopolis questionnaire responses, and in some instances, web searches. Where individual data were missing or incomplete, relevant records were eliminated from the analysis and calculations done on complete records only. 13 Our analysis excludes the separate Telecommunications Programme, which was bracketed with KTS for administrative purposes, but which has a separate budget, has been separately evaluated and, in practice, has had limited interaction with KTS. The data presented here cover the period 1998-2001. The total allocated during this time has been approximately 266 MSEK, covering 86 projects and involving at least 130 different organisations. This is made up of 145 MSEK in grants to projects 14 and 120 MSEK 15 in industry co-financing. Of the grants awarded, 143 MSEK (99% of the amount granted) were actually paid out. About 90% of the industrial contribution is made in kind (naturainsats) and 10% in cash. It is normal in this type of programme that the industrial contributions mainly take the form of personnel time. 13 14 15 An implication of our work to extend the available data set is that there can be minor discrepancies between our figures and those presented by VINNOVA in other contexts Excluding about 500 ksek spent on administrative projects This is a minimum value. Where figures are not available, we have not made any estimate of industry s contribution 13

Exhibit 2 shows the distribution of funds allocated across technical areas, with Systems Development projects accounting for 50% of total funding. The Modelling and Simulation area has the second largest spend, accounting for 17% of total funding. Exhibit 2 Programme Spend by Technical Area (total 266 MSEK) Modelling and simulation 19% Horizontal projects 1% Optimisation of infrastructures 3% Systems and decision support for large-scale operations 14% Systems Development 49% Systems Architecture 14% In Exhibit 3 we explore the sub-set of projects for which we have data about industrial co-financing. The Systems Architecture sub-programme inherited a handful of projects from NUTEK s earlier, 100%-funded Embedded Systems programme, which explains the comparatively low level of industrial funding for this area. Industrial spending on horizontal projects is negligible (118 KSEK), accounting for only 5% of the total spending in this area. This suggests an important tension between the programme s goal to generalise knowledge about complex systems across different applications and companies desire to tackle specific problems within individual areas. In effect, generalising knowledge would amount to the creation of a public good so there is no industrial interest in funding it. Instead, we see the classic market failure associated with research. 16 16 Arrow, Ken (1962) Economic Welfare and the Allocation of Resources for Invention, in Richard Nelson (editor) The Rate and Direction of Inventive Activity (Princeton University Press) 14

Exhibit 3 Funding sources by technical area 17 Horizontal projects Optimisation of infrastructures Modelling and simulation Systems and decision support Industry Government Systems Architecture Systems Development 0 10000 20000 30000 40000 50000 60000 70000 KSEK NB - there are 63 projects counted since 23 of the original 86 are excluded. Of these 1 is an evaluation, 1 is a programme conference, 4 have no funding data and 17 have no industrial funding data. Exhibit 4 shows the size distribution of the projects in terms of project funding. Almost three quarters of the projects have total budgets of less than 5 MSEK, implying that the component of these projects that funds research at a university is at most 2.5 MSEK. At this maximum, this is barely enough to fund a single technical doctorand for the period of the projects. With the exception of a handful of projects, especially the horizontal ones added late in the programme, projects were generally planned to last 36 months, but were shortened to 32 months during 2000 as a result of budget difficulties at NUTEK, and their subsidy budget correspondingly reduced. However, the size and shape of project funding confirms that KTS projects tend to involve a mix of industrial and university work, and are quite distinct from the model promoted by the Wage Earner Fund foundations, which focus on individual doctorands, with industry participating more as an observer. We can conclude that the great majority of projects involved rather fragmented funding for the universities. A few handfuls have had more significant funding. This has implications for project critical mass and interdisciplinarity. 17 For 4 projects in the data set, no data on funding was available. These have been excluded from the analysis (resulting in a total of 80 projects) 15

Exhibit 4 Size of Projects 14 12 Number of projects 10 8 6 4 2 0 <1 1-2 2-3 3-4 4-5 5-6 6-7 7-8 8-9 9-10 10-11 11-12 MSEK NB - there are 63 projects counted since 23 of the original 86 are excluded. Of these 1 is an evaluation, 1 is a programme conference, 4 have no funding data and 17 have no industrial funding data. Exhibit 5 below indicates the composition of the programme by technical area. The 83 projects involve 165 organisations in total, so that projects have an average of three participants. The majority of projects (82%) have 4 or fewer participants. The largest number of participations in a project was 10. Projects with only one participant tended mostly (65%) to be at universities. On average, the Systems and Decision Support projects are the most networked, although the Systems Development Area actually has a bigger absolute number of large projects (Exhibit 6). The horizontal projects are run 50/50 by academics and consultants. Most are single-contractor projects. In one case, seven universities got together to write a book on software engineering. Consistent with the financing picture, there has been no real industrial interest in doing these projects. The 84 different projects have 27 different organisations in the role of project leaders. Sixteen project leader organisations lead one project only most of them industrial companies. Despite the programme board s strong desire that projects should be tied to industrial needs, universities, especially the well-established southern Swedish technological universities with strengths in control and software, dominate the picture of project leadership (Exhibit 7). 16

Exhibit 5 Technical Areas in Detail Technical Area Number of Projects (No. of these projects cancelled) Number of Organisations involved Number of Participations 18 Average number of participations per project Systems development 34 (3) 70 114 3.26 Systems architecture 14 21 33 2.36 Systems and decision support for large-scale operations Modelling and simulation Optimisation of infrastructures 9 (1) 31 37 4.11 13 (1) 24 38 2.92 4 9 13 3.25 Horizontal projects 9 (1) 10 16 1.78 Total 83 130 251 2.99 *Understated, owing to missing data Exhibit 6 Participation Trends across Technical Area Systems Development Systems Architecture 12 12 10 10 8 8 6 6 4 4 2 2 0 1 2 3 4 5 6 7 Number of participants in a project 0 1 2 3 4 5 6 7 Number of participants in a project Systems and Decision Support for control of large-sclae operations 12 Modelling and Simulation 12 10 8 6 10 8 6 4 4 2 2 0 1 2 3 4 5 6 7 Number of participnats in a project 0 1 2 3 4 5 6 7 Number of participants in a project 18 Column shows the number of organisations involved in a project. For example, a project with 4 different organisations involved has 4 participations; these 4 organisations can of course be involved in more than one project in the area, therefore can have multiple incidents of participations 17

Optimisation of Infrastructres Horizontal Projects 12 12 10 10 8 8 6 6 4 4 2 2 0 1 2 3 4 5 6 7 Number of participants in a project 0 1 2 3 4 5 6 7 Number of participants in a project Exhibit 7 Most Dominant Project Leaders LTH KTH LiU CTH SU LTU UU Basesoft Open Systems AB HiS SAAB AB SICS 0 2 4 6 8 10 12 14 16 18 Number of projects led Basis: Project leader organisations in charge of 2 or more projects We were interested in the role of large companies in the KTS programme, especially because there has been a strong trend in policy through the period of the programme to focus support on SMEs and on networks of participants. KTS clearly follows the networking trend, but the importance of large firms in the programme is much more in the style of programmes contemporary with the first KTS programme than programmes being launched today. Exhibit 8 shows that large firms (defined here as those with more than 500 employees) are important in most areas of the programme, especially the Systems Development area, which is the one that receives the most funding. Small firms have no presence in Area 6, optimisation of infrastructures. Overall, large firms have 2.6 times as many project participations as small ones. 18

Exhibit 8 Participant Type by Technical Area Horizontal projects Optimisation of infrastructures Modelling and simulation Systems and decision support for control of large-scale operations Small Large Uni Systems Architecture Systems Development 0 10 20 30 40 50 Number of participations Exhibit 9 shows the most frequently participating organisations in the programme as captured through the number of participations. The major Swedish systems builders are strongly represented. The process industry is poorly represented, and this has been of concern to the programme management. However, it is worth noting that the process industries both have a low ratio of R&D to sales and tend to obtain their capital-embodied technologies from external suppliers, rather than developing them in-house. Ericsson and ABB are among the most frequent participants, though neither features in the programme as a project leader. Rather, they work through relationships often long established with university partners. No small firms feature in this list. Exhibit 10 shows the dominant universities and firms by technical area, suggesting a moderate degree of specialisation though both KTH and Lund are broadly represented across the programme (by different university departments). 19

Exhibit 9 Most Frequent Participants in KTS Organisation Number of participations Organisation Number of participations Ericsson 20 Sydkraft 5 ABB 18 FMV 4 KTH 18 UU 4 LTH 18 Volvo 4 LiU 15 Assi Domän 3 CTH 13 LTU 3 SU 8 Telia Research 3 SAAB 5 Basis: Organisations with 2 or more participations in the programme Exhibit 10 Dominant Organisations by Technical Area Technical Area Dominant Universities Systems development LiU (8), CTH (6) LTH (6), SU (6) Dominant Firms ABB Corporate Research (5), Ericsson Mobile (3), Telia Research (3) Systems architecture KTH (5), CTH (4) ABB Robotics (2) Systems and decision support for control of large-scale operations KTH (2), LTH (2) FMV (2) Modelling and simulation KTH (4), LTH (3) ABB Corporate Research (3), Assi Domän (3) Optimisation of infrastructures LiU (2) n/a Horizontal projects LTH (2) Lawson Konsult (2) The earlier analysis shows that the programme works through small networks, with the average project having three participating organisations. But KTS also forms a higher-level network, connecting together significant parts of Swedish industry and academia concerned with system building. Potentially, therefore, it provides an instrument for transmitting and sharing learning across a significant part of the industrial economy. 20

21 Exhibit 11 Disaggregated Network Analysis LTH SU KTH CTH LiU Whirpool Sverige Electrolux FMV Telia Research Karlstad Sony Ericsson Sydkvaft Sycon Energikonsult Kockums Focal Point Telelogic ABB Corporate Research Assi Domän ABB Robotics Atlas Copco Rock Drills ABB Automation Products Ericsson Mobile Association of Sweedish Engineering Industries 3 3 3 4 4 4 Exhibit 12 Aggregate network analysis LTH SU CTH KTH LiU Whirpoo lsverig e FMV Electrolu x Sony Ericsso n Sydkvaf t Sycon Energikonsu lt Kockum s Focal Point Telelogi c 3 3 SAAB 4 Ericsso n Associatio nof Sweedis h Engineerin gindustrie s ABB Atlas Copco Rock Drills Volvo LTU Assi Domän Karlsta d Telia Researc h 4 3 4 5 3 3 3

We analysed programme participation by organisations at both an aggregated and a disaggregated level. For companies with many subsidiaries, frequent interactions with other organisations are sometimes only apparent at the aggregated (group) level. For example, the high frequency of interaction between CTH and ABB is only apparent at an aggregate level of analysis rather than at the subsidiary level (such as CTH interactions with ABB Robotics, ABB Automation Products or ABB Corporate Research). Exhibit 11 and Exhibit 12 depict the inter-organisational relationships involved. 19 At both a disaggregated and an aggregated level of analysis, LTH appears to be the most highly networked university. The prominence of other universities such as CTH, LiU and SU can be seen at the aggregate level of analysis. It appears from these Exhibits that KTS has succeeded in building a national arena in which to explore complex systems technologies. The universities form important nodes, because they can play a role in spreading any generic knowledge that is produced. The KTS network spans several different types of system, from defence to domestic appliances. As is usual in networking, many of the individual network links are old. For example, the control engineering researchers have been working with control specialists at ABB (formerly SATT Control and later Alfa-Laval Automation) at least since the IT4 programme in the late 1980s, and KTS has captured many such links. A key question for KTS is: Has the programme been able to cause additional things to happen within this larger network, which would not in any case have happened? The acid test will be the extent to which the programme has been able to interconnect the knowledge of the project participants in the larger, programme-level network. 3.2 What the Participants Did In this section, we use part of the participant questionnaire to explore the reasons why people and organisations participated, and what they hoped to gain. The participants characterise their projects as being in the core of their business, and as offering somewhat incremental improvements. While the complexity label sounds dangerous, in fact tackling it is not particularly risky. The participants aim to learn about new techniques, tools, methods and skills, and to access the complementary assets of partners. These people are in many cases not pushing the envelope. They are doing a little bit more of the same. The activities are likely to be additional, in the sense that they would not have happened without subsidy, but they reflect the high degree of industrial influence over project selection. As the previous chapter indicated, the KTS programme is only one chapter in a much longer set of relationships and technology programmes. Overall, precisely half (50%) of the projects followed on from or exploited work in earlier STU or NUTEK-funded projects. Many of the network relationships involved are similarly stable. Most participants considered the strategic importance of their project to be high (Exhibit 13). There was no meaningful difference in the projects perceived 19 Diagram shows interactions between organisations for which the number of interactions is greater than 1. Lines represent 2 interactions unless they are otherwise labelled in bold 22

importance between industry and academic respondents. We need to interpret this finding carefully. The questionnaire went to R&D people, in both industry and academia. While these have different purposes, they also have a culture and many values in common. Certainly we will get different answers from the industrial R&D staff than we would obtain from corporate management. We need, therefore, to read these responses together with information about how the participants especially those in industry actually behaved. In this case, the responses tell us that the KTS projects are quite central to what respondents normally do. Since the participants also see their projects as comparatively long term, their strategic importance relates primarily to the R&D function, rather than necessarily to the business as a whole. Exhibit 13 Strategic Importance of Projects (n=74) Strategic Importance of the Project 50% 40% 42% 30% 20% 24% 28% 10% 0% 1% Minor 4% Major We asked respondents to describe their projects in terms of dimensions such as cost, risk and so on. Exhibit 14 summarises their answers, and shows that most projects were considered to be exciting, technically complex, long-term efforts undertaken in core technology areas for the participating organisations. The distributions of responses that lie behind Exhibit 14 are shown in the Appendix. Most of the distributions have distinct peaks. For example, in the distinction between low and high cost, almost 50% of the respondents put their project in the middle of the scale: neither one nor the other. While the projects were complex, they were necessities and rarely high risk, dealing with core technologies that were strongly connected to other in-house work. These characteristics are consistent with the view we have formed of the projects as being rather strongly driven by industry, which does not like uncertainty. The projects were long term and while they were mission oriented and applied they were much more likely to lead to benefits for R&D than production. They tended to be about processes rather than products, and were much more likely to involve software than hardware. Project ideas have been generated by both the industrial and research communities, and there is a balance between projects oriented to generic issues and projects oriented to specific problems. The programme therefore seems to have achieved its objective of doing industrially driven work, while still exploiting academic input. 23

Exhibit 14 Nature of Projects (n=77) Mundane Technically simple Hardware-oriented Exciting Technically complex Software-oriented Short-term In a peripheral technology area for your organisation A luxury Production-oriented Product-oriented Not connected to other inhouse projects Curiosity-driven High risk High cost Fundamental Oriented to a specific complex problem Idea came from industrial community Long-term In a core technology area for your organisation Necessary R&D-oriented Process-oriented Strongly connected to other inhouse projects Mission-oriented Low risk Low cost Applied Oriented to generic issues of complexity Idea came from research community 1 2 3 4 5 None of the respondents judged their work to be technically simple, but a significant proportion of technically complex projects was still judged to be low risk (Exhibit 15). Tackling complexity, in others words, is in many cases rather a routine matter. The technical interest seems to be in incrementally improving the way complexity is handled, rather than using the subsidy and leverage available from the projects to take bigger risks than companies can normally take. 24

Exhibit 15 project Comparison of perception of risk with technical complexity of 16 14 12 Risk vs. technical complexity(n=76) high med low Frequency 10 8 6 4 2 0 Technically simple Technically complex In order to benchmark aspects of KTS, we have compared the responses to some questions in our KTS survey to responses to similar questions in other evaluation surveys. Exhibit 16 presents the combination of the mean average response to the question exploring the type of projects undertaken in five large Advanced Technology Programmes 20. Comparison with the KTS data (presented as a separate plot line) illustrates the striking similarity of many of the responses. 20 The UK Alvey Programme, the Swedish IT4 Programme, The Finnish Electronics Design and Manufacturing and Electronic Publishing and Printing programmes and the EU Climate Change Programme 25

Exhibit 16 Comparison of programmes analysed in ATP review with similar responses from KTS programme 5.0 4.5 ATP KTS 4.0 3.5 3.0 2.5 2.0 1.5 1.0 Low Cost(1) High Cost(5) Low Risk(1) High Risk(5) Trivial(1) Complex(5) Mundane(1) Exciting(5) Necessary(1) Luxury(5) Short-term(1) Core(1) Long-term(5) Peripheral(5) Singular(1) Collaborative(5) The following two charts illustrate the similarity of responses between the IT4 and KTS programme surveys, when comparing the duration of projects with the technology areas explored the nature of R&D undertaken. IT4 was the Swedish national research programme in industrial IT, conducted in the late 1980s. Some of the R&D partnerships and some of the key industrial figures from the IT4 programme have survived the intervening decade to the second KTS programme. IT4 is quite a typical pre-competitive, collaborative R&D programme. The similarity between the foci of respondents work in the two programmes is striking. Participants are rarely working in non-core technologies and are usually engaged in longer-term projects. KTS conforms to type. 26

Exhibit 17 The Nature of R&D in the IT4 programme NUMBER OF PROJECTS 25 20 15 10 CORE TECHNOLOGY 5 0 1 2 3 3 4 5 LONG TERM 4 2 PERIPHERAL TECHNOLOGY 5 1 SHORT TERM Exhibit 18 The Nature of R&D in the KTS programme Number of Projects 25 20 15 10 5 Long term 0 Core Peripheral Short term Respondents were asked to indicate the importance of different types of goals for their own organisation s participation in their KTS project. The ten most important motives and goals overall are shown in Exhibit 19. The Exhibit also shows separately how knowledge infrastructure respondents and industrial respondents ranked these goals. 27

There are some obvious and expected differences between the university and industry responses. Industry is looking for new university relationships. Universities are not (which may be a pity in Sweden s rather fragmented university system). Process R&D and acceleration of R&D are more important to industry than to the academics. But both place a lot of emphasis on understanding and learning about tools and methods, and on accessing complementary assets, especially expertise. The academics are more interested in extending their work with their existing knowledge base than in getting into new areas. Overall the picture is of consolidation and incremental improvement, rather than a radical seeking for new types of knowledge. This is not necessarily a bad thing, but it is more likely to serve the part of the KTS agenda relating to applications than that which aimed to improve general knowledge about complex systems development. Exhibit 19 Top 10 Motives and Goals (n=74) Motive / goal All responses Research responses Industry responses Increased understanding of new methods and tools 1 1 1 Enhancement of existing knowledge base 2 2 7 Development, evaluation or improvement of tools and 3 6 3 techniques Increased skills of research staff 4 3 10 Access to complementary sources of expertise 5 5 4 Formation of new research partnerships and networks 6 4 8 Better co-operation with universities and research 7 13 2 institutes Acceleration of R&D 8 9 5 Enhancement of knowledge base in new, alternative 9 8 9 technology areas Development or improvement of new processes 10 12 6 The Exhibit highlights the importance of knowledge-oriented goals to all participants. Industrial participants tended to rank networking goals as slightly more important than did the knowledge infrastructure participants. Certain exploitation-oriented goals, such as the development of new or improved tools, techniques and processes were also ranked highly, especially by industrialists. 28

4 Results of the Programme Our questionnaire analysis suggests that participants did well in satisfying their own goals through the programme, but is less clearly positive about the projects effectiveness in reaching programme goals. Most of the projects were additional and there appears to have been little free riding in the programme. The main outputs were knowledge and networking. Despite the long term nature of the projects, some short term commercial exploitation had already been achieved. New knowledge was generated but externalities from the projects, in the form of knowledge and capability transfers, had been limited. Both industrial and academic participants felt that the benefits of being involved in the KTS programme had very significantly outweighed the costs. The overall picture, then, is consistent with the impression we have already formed of projects that have strong industrial steering but a weaker link to wider social and technical goals. Programme management collected a number of simple indicators of knowledge generation and transfer. These are summarised at the end of the chapter, confirming that the programme produced a significant number of higher degrees as well as other, commercially useful outputs. 4.1 Questionnaire Analysis 4.1.1 Additionality Additionality is an important test for any state intervention. It would be wasteful to spend taxpayers money on subsidising R&D that would have been done, regardless of whether the subsidy were available. It is, of course, rather hard to establish whether this is the case, since we cannot re-run history without the KTS programme and look to see whether the same kinds of projects are performed. Asking participants directly about additionality provides a partial answer though we should only accept it if it is consistent with other aspects of project participants behaviour. Respondents were asked to consider what would have happened if their project had not received funding through the programme. Exhibit 20 shows that most respondents (58%) believed the project would not have gone ahead at all if funding from NUTEK had not been available (full additionality). Just over a third (39%) stated that they would still have undertaken the project but with reduced objectives, partners or funds or over a longer time scale (partial additionality). A small minority (3%) indicated that their project would have gone ahead unchanged if funding from the KTS programme had not been forthcoming, suggesting no additionality in these cases. These are free riders and while free riding is a waste of tax payers money, the level involved is, according to the participants, rather low. 29

Exhibit 20 Additionality (n=77) Project would have gone ahead unchanged 3% Project would have gone ahead but in different form 39% Project would not have been undertaken at all 58% We would expect projects in academia to be highly additional, but the academics do appear to have other financing possibilities. 36% of them said that without NUTEK funding their project would have gone ahead in another form, while 64% said the project was completely additional. In industry, there was also a majority (54%) who said that the project would not have happened without the financing from NUTEK. All those confessing to being free riders were in industry (8% of industrial participations still an apparently low level). A key part of the traditional argument for state support of R&D is that it reduces risk and uncertainty. Even if R&D projects might in general show a good return on investment (and there is plenty of evidence that this is the case), companies can be reluctant to invest in individual research projects because if the risks involved. In high risk projects (rated 4 and 5) the majority of KTS participants indicated that the project would not have proceeded without funding from the programme. As the perception of risk diminishes, so the likelihood of continuing the project at least in some other form - increases. In around half of the low risk projects, work would have gone ahead in some other form. The free riders are doing low risk projects, as we would expect. Even for low risk projects, however, almost half of the respondents claim that their work would not have gone ahead without support. This analysis, therefore, broadly supports the response we obtained to the direct question about additionality, confirming the case for state intervention. 30

Exhibit 21 Does the risk involved in the project affect the likelihood of work going ahead with funding from the Programme? 20 18 16 14 12 10 8 6 4 2 0 Impact of no funding vs. risk (n=76) high medium low Unchanged Not at all Gone ahead We asked respondents about the extent to which the outputs and outcomes from the projects had met their expectations. The results (shown in detail in the Appendix) indicate a generally good level of goal attainment, with expectations being met in 47% of cases and surpassed 32% of the time. Exhibit 22 plots the results. Exhibit 22 Goal Attainment (n=74) All responses Less than As expected More than Total expected expected Low importance 12% 20% 2% 34% Medium importance 3% 13% 4% 21% High importance 5% 14% 25% 45% Total 20% 47% 32% 100% The situation looks even better when we consider only those goals that were rated as important to participants. In over half (57%) of instances, expectations were surpassed and expected outcomes were met in a further third (31%) of cases. In only 12% of cases participants indicated that they had achieved less than hoped for in relation to an important goal, and there is no obvious single expectation that was consistently under-delivered. The areas in which the most participants had exceeded their expectations in the project were Increased understanding of new methods and tools Formation of new research partnerships and networks Access to complementary sources of expertise 31

Better co-operation with universities and research institutes The last of these produced especially positive results from industry. This fits well with the more general pattern of goal attainment in advanced technology programmes, which show that knowledge and networking are key benefits of participating in statefunded R&D programmes. The knowledge focus of the programme is supported by the fact that 63% of respondents exceeded their expectations of increasing their understanding of new tools and methods while a rather lower 31% had exceeded their expectations of actually developing, evaluating or improving such tools. 4.1.2 Factors Affecting the Projects We asked participants to indicate the extent to which various factors had impacted upon the progress of their project. Exhibit 23 lists the top 5 factors rated as exerting a positive impact. The competence of the participants, the structure of the project, and the adequacy of goal specification and outputs had each facilitated progress in the vast majority of cases. Exhibit 23 Factors Affecting Project Progress Top 5 Positive Factors (n=75) % of projects affected positively Competence of own organisation 92% Competence of other participants 88% Adequacy of project outputs 81% Structure of the project 81% Adequacy of goal specification 80% By comparison, few factors caused problems for significant numbers of projects. The most commonly cited problem related to the premature curtailment of funding, which had hampered progress in just under a third of cases. The process of drawing up consortium agreements, delays in technological developments within the project and availability of qualified personnel had also caused problems for a minority. Exhibit 24 plots the results. Exhibit 24 (n=75) Factors Affecting Project Progress Top 5 Negative Factors % of projects affected negatively Access to sufficient amounts of project funding 27% Delays in technology development within the project 19% Availability of qualified personnel 15% Process of drawing up consortium agreements 14% Restructuring / strategic changes in project partners 13% Respondents were also asked to indicate the impact various factors had exerted on their ability to exploit project results. Exhibit 25 plots the results and indicates that competence and interest levels within their own organisation, adequacy of project outputs, the structure of the project and competence levels within other organisations were seen to have exerted a strong positive influence on exploitation in most cases. 32

Exhibit 25 Factors Affecting Exploitation Top 5 Positive Factors (n=75) % of projects affected positively Competence of own organisation 68% Adequacy of project outputs 64% Competence of other participants 59% Levels of interest within own organisation 57% Structure of the project 55% The factors that exerted the strongest negative influence on project exploitation are shown in Exhibit 26 below. It reveals that (a lack of) project funding hampered participants ability to exploit results in around a quarter of cases (26%). We interpret this as a further complaint about the reductions in project funding made at the end of the programme. That this proportion is not higher is probably connected to the focus of the participants on knowledge and networking rather than more concrete product and process results. Delays in technological developments within the project, lack of qualified personnel and restructuring within partner organisations also caused problems. The complexity of the technical issues involved also hampered exploitation for around 1 in 8 participants. Exhibit 26 Factors Affecting Exploitation Top 5 Negative Factors (n=75) % of projects affected negatively Access to sufficient amounts of project funding 26% Delays in technology development within the project 16% Availability of qualified personnel 14% Restructuring/strategic changes in project partners 14% Complexity of technical issues addressed 13% 4.1.3 Importance of Outputs We are interested in participants views about what project outputs are important, because this gives us a better insight into their motivations. Respondents were therefore asked to indicate the importance of various types of outputs in assessing the success of their project. The proportion of respondents rating each output as essential or important is shown in Exhibit 27. The results indicate that both academic and industrial communities are agreed on the most important output from these projects new methods and tools, while the earlier analysis suggested that the most important achievement of the programme was improved understanding rather than to methods or tools themselves. As one would expect, almost all knowledge infrastructure participants cite publications and PhD theses as important outputs but only around a half of industrial participants feel the same. 33

Exhibit 27 Importance of Outputs (n=49) All responses Knowledge Industry only infrastructure only New methods or tools 95% 95% 94% Publications in refereed journals 74% 93% 52% Other publications 72% 85% 56% PhD theses 68% 85% 45% Pilots or prototypes 65% 65% 65% New processes 64% 70% 58% New products 32% 23% 41% Norms and standards 23% 23% 24% Patent applications 19% 26% 12% Patents granted 13% 18% 6% Other 8% 7% 8% The number of respondents behind Exhibit 27 is limited, so we should be cautious about over-interpretation. It is nonetheless interesting that the academics are more interested in patenting than the industrialists, perhaps reflecting the fact that they are likely to own the intellectual property rights arising from the projects (lärarundantaget). 4.1.4 Contribution to Programme Goals Respondents were next asked to indicate the extent to which their project had contributed towards a number of programme-level goals. The results are shown in Exhibit 28, which shows that according to the knowledge infrastructure, KTS projects have significantly enhanced systems development capabilities in industry and academia, and have supported the development of new tools and methods. The responses from industrial participants are generally less optimistic and suggest that the projects have had only modest impacts overall on systems development capabilities. Few projects were rated as having supported significantly the transfer of capabilities and technologies between disciplines and sectors. 34

Exhibit 28 Contribution to Programme Goals (n=76) Contribution to Programme Goals Industry Research Base Improved systems development capabilities within the research base Improved systems development capabilities within industry Development of tools and methods for complex technical system development Dissemination of information on new methods and tools via demos / apps Transfer of capabilities between disciplines and sectors Transfer of technologies between disciplines and sectors 1.00 No 2.00 3.00 4.00 Major 5.00 contribution contribution Percentage of Respondents We need to be careful in interpreting responses about own achievements. In other studies, we have been able systematically to compare experts (peer reviewers ) judgements about projects with project leaders own judgements, and found the clear and positive bias one would respect in project leaders self-assessments. We are therefore inclined to mild skepticism about the judgements made by respondents in Exhibit 28 and to consider these values as upper bounds. Even without applying such a discount, the industrial project leaders seem to be telling us that their performance in tackling programme goals is only middling (lagom). 4.1.5 Commercial Exploitation Industrial respondents (only) were asked to indicate whether their project had already led to commercial benefits for their company. Just over one-third (34%) stated that commercial gains had been realised. The nature of the benefits varied, but tended to be savings to production costs gained through improvements to development lead times, improved testing capabilities, reduced processing costs, better quality control, improved process management, and so on. Most respondents were unable to quantify the benefits, but two companies have realised savings of 20 MSEK and 15 MSEK respectively and one further company is making savings of around 1MSEK per year. Industrial respondents were also asked to comment on their plans for future commercial exploitation of project results. Almost half (46%) indicated that they have concrete plans to exploit the project results in the future, though several other respondents stated that premature curtailment of project funding meant that commercial exploitation remained beyond their reach. 35

Overall, 60% of industrial participants had already realised commercial benefits, or expected to do so in the future. 40% of industrial participants indicated that they did not expect any commercial benefits from the project at any time, past or present. 4.1.6 Self Assessment We asked project leaders to rate their projects against a number of standard performance criteria, and the results are plotted in 36

Exhibit 29. Most rated their project fairly highly in most respects, though at an aggregate level the highest ratings were assigned to projects in terms of the Adequacy of the project agenda / work plan Quality of the project outputs Extent to which the project had significantly contributed to existing knowledge In previous studies, where we have compared self- and expert evaluations for the same projects on these kinds of dimensions, the two have tended to correlate well, but with a positive bias of up to 1 in the self-evaluation scores. Here, we have no expert control, but the reader may find instructive the thought experiment of applying a systematic discount to the self-evaluation scores in 37

Exhibit 29. Lowest ratings were assigned to projects in terms of the extent to which participants had exploited results, and in terms of their utility and uptake by organisations outside the project team. Overall, knowledge infrastructure participants assigned higher ratings than did industrialists. The most marked differences between the two communities concerned the extent to which projects technical and exploitation goals had been attained. 38

Exhibit 29 Self Assessment (n=75) Self Assessment Industry Research Base The adequacy of the project agenda / workplaan The adequacy of the project resources in terms of staff, equipment, finances, infrastructure, etc. The organisation and management of the project The quality of the project outputs The extent to which the project has made significant contributions to knowledge The extent to which the project's technical and exploitation goals were attained The utility of the project results to your organisation The extent to which the project has been exploited within your organisation The utility of the project results to other business / research institutions The extent to which the project has been exploited within other businesses / Ris 1.00 Very low / 2.00 3.00 4.00 5.00 Very high / poor good 4.1.7 Costs and Benefits Respondents were asked to indicate how the costs and benefits of participation in the KTS programme compared. The results are shown in Exhibit 30. For the vast majority of participants the benefits have far outweighed the costs. The knowledge infrastructure has derived a slightly more positive outcome overall, but this is not unusual in cost-shared RTD programmes where (a) the industrial community bears a significant proportion of the costs and (b) industrial benefits tend to be realised after project completion. 39

The profile below is typical of that found in successful RTD programmes, and indicates a good level of performance for the programme overall, from the participants perspective. Exhibit 30 Costs and Benefits of Participation (n=74) Costs and Benefits 40% 35% Industry Research Base Percentage of Respondents 30% 25% 20% 15% 10% 5% 0% 1 2 3 4 Costs equal 5 6 7 benefits Costs outweigh benefits Benefits outweigh costs 4.2 Results Reported by Projects NUTEK/VINNOVA programme managers have continuously monitored the achievements of the programme in terms of a handful of quantitative indicators. This has been done at the programme s initiative and through ad hoc contacts with the projects, rather than as part of a wider administrative routine within the agency. Exhibit 31 Key Output Indicators Higher Degrees Awarded Publications Industrial Outputs 38 licenciates 24 PhDs Source: VINNOVA 209 reports 86 scientific articles 18 books 28 new methods 28 prototypes 20 new products 8 new processes 6 patent applications 3 start-up companies 40

5 Management and Administration NUTEK had well-established routines for programme definition and management, which in general performed well. These were documented in a benchmarking report 21 commissioned by NUTEK in the mid-1990s, and have subsequently been compared with other R&D programme management practices, especially in the Nordic area. 22 We chose, therefore, in this evaluation not to attempt an exhaustive review of the programme management processes involved in KTS. Instead, we used our questionnaire, interviews and discussions with programme management 23 to look for signs of trouble, and to get an overall assessment of programme management from the user perspective. The programme board comprised experienced and influential stakeholders and provided helpful continuity with the past, a powerful knowledge base and a resource for mobilising projects. It took measures that resulted in high industrial influence over the programme and its projects. However, the great size of this second KTS programme meant that both the board and the administration at NUTEK/VINNOVA were stretched. The board stressed programme conferences as a way to bring the programme together. This was partly successful, though the wide scope of the programme proved to be a barrier to bringing together people with very different interests. The rather bottom-up way the programme had been populated with projects led to fragmentation. Generic complexity questions didn t arise spontaneously from industry or from academics. It appeared, therefore, that the programme needed both a stronger theoretical basis and more appropriate incentives in order to tackle generic complexity. Participants were generally very positive about the quality and timeliness of administration. However, from our evaluation perspective, and indeed in terms of programme management s need for basic management information and some kind of statistical process control, the programme did not have adequate Information Systems support. 5.1 The Programme Board The KTS programme board appears to have played an unusually large role in shaping the programme. Most of its members had also been members of its predecessor, so they were well placed to specify how to adjust the programme. The composition of the board was Lars Göran Rosengren, Volvo TU, (chair) Owe Borgström, Ericsson Utveckling AB Ingemar Carlsson, FMV 21 22 23 Erik Arnold and Paul Simmonds, Programme Management Benchmark Study; Final Report to NUTEK, Brighton: Technopolis 1997 James Stroyan and Erik Arnold, Comparative Study on Administrative Burdens and Rules fo Procedure between the EU Research Programmes and those of the Individual Member States, project IV/98/06, Luxembourg: European Parliament, 1998 Arne Otteblad s note Styrgruppens metoder för verksamhetsstyrning och intervenering, was especially useful 41

Hans Skoog, ABB Corporate Research Lennart Ljung, Linköpings Universitet Lena Mårtensson, Kungliga Tekniska Högskolan Gustaf Olsson, Lunds Universitet Bengt Sjögren, Competens Kraft Lagman AB While board members sit in a personal capacity and not as representatives of their organisations, much of their value is their knowledge and experience. It would not only be difficult for them to avoid acting as stakeholders but it would also be undesirable. It is not surprising that the companies represented in the board are also major participants in the programme, and with their presence comes a commitment to a number of long-standing R&D themes. This brings both relevance and potential lock-in to existing industrial organisation and priorities. We return to this issue in our conclusions. The board decided that the new programme should have a closer industrial focus than the first, so it developed a special set of appraisal criteria for project applications and tightened up the requirements for projects to be accepted into the programme. Proposals needed letters of intent from participating companies. Both these and the consortium agreements that formed the bases of project contracts had to be signed at a high level of the companies, in an attempt to secure commitment. Intellectual Property Rights (IPR) should be explicitly handled, underlining that the industrydominated board wanted to see rather short-term results from the programme. The intention was to persuade NUTEK to enter into firm, 3-year commitments to projects, but this proved to be legally impossible, given the annual system of state budgeting. The board tackled the larger scale of the second KTS programme by setting up separate sub-groups of the board and supporting project officers for each of the subareas. This allowed more focus and let individual board members expertise be exploited, but it also meant that the breadth of perspective offered by the whole board was not available to most of the application process. Only at the final stage of project approval did the whole board look at individual proposals. Some board members have argued that this reduced the extent to which cross-cutting issues such the human dimension and projects potential contribution to a meta-understanding of complexity were considered. On the positive side, making the roles of the members more specialised meant that the could act as uncles to individual larger projects, offering a degree of supervision and acting as sparring partners for the project performers. The programme put great weight on holding kick-off and then annual conferences, in an attempt to pull its rather varied activities together into a coherent whole. 5.2 Interviews with Board Members Our interviews with six of the board members indicated that the board operated socially as an unbelievably well functioning group. The members had been strongly engaged and with the exception of one academic who had not been seen for two years participated regularly in the demanding meetings of the programme. The increased scale of the second KTS programme, however, meant that the board had become overwhelmed and had been forced to break into smaller, more specialised groups supported by NUTEK project officers, in order to cope. This had made it more difficult to operate at the meta-level of generic complexity and to take account 42

of the human dimension as well as had been hoped. Some board members regretted that it had not been possible to make much of a link from the programme to firstdegree education (although it is not clear to us that this was in any way a central mission of the programme). There was a sense that the programme had been too technically oriented, driven in part by what board members saw as NUTEK s very technical brief. The roles of users and customers in complex technical systems were vital, and should be better integrated. Management of complex technical systems and their design was important, not just the hard technology. As industry moves further towards designing via platforms so it needs to take increasing account of the interactions among the systems, sub-systems and components of platforms, some of which can be unexpected. Platform management, not just the technical interaction of modules, needs to be understood better. The programme had been populated with projects in a very bottom-up way, and this had not been complemented by a sufficiently strong top down design component. This had resulted in something of a shotgun strategy, with a lack of focus on important areas. The requirement for industrial co-financing was one important reason why the programme had been so strongly steered towards fragmented, concrete applications rather than generic complexity questions. No researcher ever came forward with an idea about generic complexity, as one member put it. Projects were also too small to enable much generalisation. While there was a great deal of discussion on the first KTS programme board about what was meant by complexity, it is not clear this was fully resolved. The wider scope of the second programme led at least one member to be confused. I never understood what the programme was about. It was never clear. We never had enough discussions about this the board was always too busy. The same member criticised the board s approach as essentially atheoretical. It needed an hypothesis in order to organise the programme. Some argued that a next step could be to move more from research and towards demonstration in a limited number of areas, but this was not possible within a single three-year programme. This would address both the need for focus and the comparatively weak dissemination of results so far outside the programme. The idea of developing a science of platforms was also attractive to some members as a basis for continuation. 5.3 NUTEK Support and Procedures Questionnaire and Interview Evidence Questionnaire respondents were asked to rate the input received from programme officials throughout the lifecycle of their project. The results obtained (Exhibit 32) were extremely positive, with most respondents indicating that they had received input and that this had proved to be helpful. 43

Exhibit 32 Input from Programme Officials (n=72) 50% Prior to proposal submission 50% During project selection/negotiation 40% 30% 20% 10% 0% Unhelpful Neutral Helpful Very helpful 40% 30% 20% 10% 0% Unhelpful Neutral Helpful Very helpful 50% 40% 30% During project implementation 50% 40% 30% Follow-up and feedback 20% 20% 10% 10% 0% Unhelpful Neutral Helpful Very helpful 0% Unhelpful Neutral Helpful Very helpful Respondents were also asked to assess NUTEK procedures for making an application for research funding. Again the results were generally positive, with most participants rating the procedures and the associated documentation as clear and easy to follow. Interviews suggested that project leaders tend to have quite a personal relationship with their project officers, easing administrative relations. Respondents were split, however, concerning the speed of the process. Exhibit 33 plots the results. The evidence from the questionnaires is consistent with that we obtained at interview. People were generally complimentary about administrative effectiveness. Project officers were seen as engaged, and NUTEK s routines as efficient. More established members of the research community felt that the project officers now had less time for personal interaction with the projects, compared with the first KTS programme, and felt this was a loss. Those who had noticed that the programme was looking for commitment at a higher level within companies than is normally the case tended to feel that this had little effect, either positive or negative. 44

Exhibit 33 NUTEK Procedures for Research Funding (n=72) 50% 40% 30% 20% 10% 0% 50% 40% 30% 20% 10% 0% 50% 40% 30% 20% 10% 0% Difficult to follow Complex documentation Slow NUTEK Procedures for Research Funding Applications Easy to follow Clear documentation Quick 5.4 Programme Administration An Evaluation Perspective We found the VINNOVA programme management very helpful in supporting our work. As we indicated at the start of this chapter, programme management works at a level of good international practice. An important problem, however, is the lack of systems that collect some of the basic data needed for evaluation. KTS programme management had had to build its own, Excel-based project tracking system, in parallel with the agency s finance and project system. This is a major waste of programme management time. It also meant that there were important data gaps about who had been involved in projects. Project leader data were fairly accurate, but data about partners was patchier and often slightly out of date, with partners both joining and leaving consortia without NUTEK/VINNOVA s knowledge. Monitoring was similarly patchy, with data missing from over a third of the projects. An implication of these systems weaknesses are that evaluations take longer, and are less complete, than is desirable, and that too much of the evaluation budget is wasted on data collection and cleaning. A second implication is that the missing data lead the programme to under-state its immediate results, via the short term indicators used, such as numbers of PhDs granted. 45

6 Project Experience and Case Studies This chapter is based on 21 interviews with project participants. First, we summarise some general reactions of the participants to the KTS programme. Next, we describe a handful of cases, selected by ourselves, which are intended to communicate a more concrete sense of how some individual projects have functioned. Finally, we draw some conclusions. Discussions with project participants confirmed that many of them worked in rather long-standing networks with industry, though new relationships were also being formed. University people were key in establishing many projects, while end users were rarely involved in project definition or execution. Projects generally claimed to be developing tools and methods for complex technical systems development and many to be raising systems design capabilities in universities and industry, including through postgraduate education. Project leaders found the KTS programme rather broad and, in some cases, hard to identify with. NUTEK administration got high marks from the project leaders. The programme s complexity agenda had a range of meanings for the project leaders. They had applied to the programme because there was an overlap of interests, not necessarily because they subscribed to the KTS programme goals. The case studies showed that there is a wide range of project types. Some projects had succeeded in integrating human factors, others had not. Those that worked well had produced useful knowledge and applicable results in the form of inputs to development and changed working practices. Horizontal project cases aiming to synthesise domain-specific work in the programme in order to develop generic systems understanding suffered from the lack of generalisable results and tended to become (useful) state of the art reviews rather than true syntheses. 6.1 Project Participant Views This section describes reactions of the project participants we interviewed to the programme. We used a semi-structured interview technique, so that we could ensure both that we covered a core set of issues such as goal attainment with all our interview partners and at the same time leave enough space for the interviewees to raise issues and questions. We approached people from all parts of the programme for interview, so that we might understand something of the breadth and variety of the programme. However, this kind of interviewing cannot be statistically representative, in the way that our questionnaire is (Section 4.1). Interpreting what interviewees say can be as much an exercise in practical sociology 24 as in factual data collection. It nonetheless becomes analytically interesting to the extent that it raises issues and questions that are consistent with those from other parts of this evaluation. 24 For example, it may be necessary to discount for arrogance or narcissism. A handful of academic interviewees told us that the NUTEK staff was not intellectually up do doing its job. The main evidence offered for this was that they were NUTEK staff and not academics, as if all non-academic career choices were inferior to academic ones, and as if the skills required to run R&D programmes were so trivial as automatically to be within the grasp of any academic which they are patently not 46

Many of the interviewees were repeat customers of programmes such as KTS, stretching back to the 1980s in a number of cases. These people have long-running research and knowledge agendas, into which they creatively fit a succession of funding sources. They inhabit rather stable-looking R&D communities with well established network relations among participants, such as A cluster of robotics and control R&D activities focusing on ABB and the Technical University of Lund (LTH). In the 1980s, when first we met some of these people, SATT Control and later Alfa Laval were involved, but their interests have been bought up by ABB over the years. LTH s world class control capabilities are an important ingredient in this cluster of activity A defence cluster, focusing on software and simulation, especially for command and control systems. Here the defence procurement agency, FMV, has played a central role over the years. While several universities (especially Linköping) are involved, this cluster tends to involve a greater weight of industrial and military involvement than other parts of the programme, and therefore a smaller academic input Another cluster of software activity linking Ericsson and LiU though Ericsson s overall pattern of participation involves wide collaboration with many universities There are probably other clusters not obvious to us, but the same time, there are many newer relationships and more isolated activities, too. Nonetheless, most of the people we interviewed could trace their project s origins back to previous state-funded research and network relationships. Because of their constant search for funding and therefore their role as project entrepreneurs, university people tended to initiate many of the projects. For them, research projects are core business and they will try to recruit whoever is necessary in order to get access to funding. Industry s involvement was more ad hoc in many cases. However, there clearly are places in industry such as ABB Corporate Research, which has among its specific missions the task of linking the company s knowledge needs to the university sector where there is a systematic, longer term agenda to work with academic partners. Smaller companies seem to become involved in order to do development or other close-tomarket work. There was very little end-user involvement in setting up or doing projects. Technical departments of system-building companies tended to see themselves as adequate proxies for customer involvement. This fact sat rather uncomfortably next to the increasing interest among participants in systems engineering, with its heavy focus on users and needs definition in the design process. Different projects made different contributions to reaching the programme goals Improved systems development capabilities within the university / research institute sector. Most projects claimed to make a contribution here, but on the basis that systems were essentially the things that the university partners already did. A minority claimed that systems work was the antithesis of the rather specialised approach followed by many of the academics, and that universities were poor places in which to do the kind of cross-cutting work envisaged in the KTS programme 47

Improved systems development capabilities within industry. Many of the projects claimed to provide this Development of tools and methods for complex technical system development. Most projects claimed to make a contribution to this agenda Dissemination of information on new methods and tools via demonstrations / applications. Our interviewees were less convinced that this had been effective, especially outside their immediate research communities Transfer of capabilities between disciplines and sectors and transfer of technologies between disciplines and sectors. These goals were the least satisfied The projects that showed the most promise for cross-disciplinary working and tackling more generic aspects of systems complexity were among the larger ones. These had the scale necessary to bring together different people and organisations. Postgraduate education was an important part of many projects, but none was big enough or long enough to fund complete PhDs. As with most research, there would be little direct trace of the work in first-degree course work, except in one or two very applied areas, where the case materials from the research could be used in teaching. There was a stronger link through final year projects (ex-jobb). If anything, there was unexploited potential to make greater use of final year students in less critical parts of projects. Almost everyone we talked to said that the scope of the KTS programme was too broad. Those who had been involved in the first KTS programme felt that it had been a mistake to make the second one so much bigger and wider. No one felt at home in the new wider definition, where so much of the programme was technically uninteresting to them. This was reflected in the programme conferences KTS held annually. These were a good idea in principle. Those who liked them in practice belonged to large enough clusters of related activity to allow them to find activities of technical interest to them. Those who found themselves without such a peer group found the conferences boring. One went so far as to say that he attended only because he was obliged to. Most of the comments about the administration were positive, with many comparing NUTEK s administration favourably with that of the European Commission. Not everyone had noticed that the KTS programme was trying to obtain commitment at higher levels to project investments from industry. In a handful of cases this had introduced delay, but few felt that the change in procedure had made a great difference. Those academics used to a 100%-funded model, where industry was incited to express an interest and perhaps participate in a reference group, felt that the old arrangement made it easier to launch projects because middle-level management could give approval. Quite a number of interviewees found the notion of complexity to be problematic, or in several cases fluffy. Some felt it was window-dressing, necessary for NUTEK to persuade the political level of the need for funding, but that it had no operational meaning. For these people, KTS was most clearly just another source of money to pay for their existing project agenda. Most believed that complexity was both intellectually and industrially important, but were more cautious about the programme s effects on these at the overall level. There was a need for a continuing 48

programme to tackle complexity, but it should be narrower and have a clearer intellectual focus. 6.2 Case Studies The main purpose of this section is not to evaluate but to explain how parts of the KTS programme operate. During the evaluation we conducted seven small case studies to illustrate the mechanisms through which the programme produced results and, if appropriate, to show the kinds of difficulties which can get in the way of such achievements. The cases are based on a combination of project participant interviews and review of background documentation. They are intended to reflect some typical examples of projects that were funded from the KTS-programme. As one might recall the overall objective of the programme has been to find general methods, tools, descriptive models etc to develop, manage, produce and use complex technical systems and master the negative aspects associated with complexity. Six different projects and seven cases were chosen and some key characteristics, in terms of the main cognitive problems addressed and the type of R&D activity of the projects are highlighted. All of the cases involve some degree of uncertainty, which is tackled through R&D and various instruments as shown in the table. The Sensor Data project is probably the most traditional in its focus on engineering applied research in a close and long standing industry-university setting as that between the department in Lund and ABB. The configuration management project is the smallest of the chosen cases and has an emphasis on trying to synthesise existing knowledge in the field. 49

Exhibit 34 Case DELTA (Volvo Parts; the Military High Command) A conceptual framework for design (2 cases) Sensor data integration and force control Sensor-data fusion for autonomous robots Configuration management Systems engineering of complex systems Some Key Characteristics of the Cases Total Budget MSEK (% industry funds) 3.052 (55%) 11.3 (63%) 5.477 (37%) 2.744 (53%) 0.3 (0%) 0.375 (0%) Main Cognitive Problem Addressed Type of R&D Activity Participants Non-deterministic or poorly understood relations among systems components and sub-systems Inability to communicate among disciplines, causing difficulties in understanding complex systems Non-deterministic or poorly understood relations among systems components and sub-systems Inability to communicate among disciplines, causing difficulties in understanding complex systems Combinatorial complexity Need to integrate across multiple systems and technologies /and therefore disciplines) Navigation and cleaning strategies for autonomous vacuum cleaners Combinatorial complexity Need to integrate across multiple systems and technologies /and therefore disciplines) Lack of widespread understanding of systems engineering principles Workshops, PhD student Masters course Ttransfer workshops PhD studies Action research Transfer workshops Applied research Experimental development Technology transfer Experimental development PhD studies Synthesis work presented at various workshops Review of KTS projects Introductory handbook on systems engineering Seminars U Gothenburg Dept of Informatics, Volvo Parts, Military High Command, FMV, Aerotech Telub AB U of Linköping, Ericsson Utveckling, Whirlpool, CelsiusTech LTH, ABB Robotics KTH Centre for Autonomous Systems, Electrolux LTH Syntell, Lawson Konsult 50

6.2.1 Project Delta The gap between the strategic thinking and the outcome of the development processes has been documented repeatedly for many years in the literature as well as in practical applications. According to one of the participants, Misunderstanding is the normal basis for systems development. The military has a long and glorious history of disastrous Information Systems development, as does industry and civil government. As people try to build very large systems, the cost of mistakes in wasted money and in failure to deliver the needed performance becomes huge. The DELTA project builds on previous work in the KTS programme s first period. The project champion was Håkan Enqvist at Aerotech Telub. Enqvist managed to bring together SKF, Volvo Parts, Aerotech Telub, the Viktoria Institute, University of Gothenburg and the Swedish military high command to set up the project. There were already well-established contacts between Enqvist and these various organisations, thereby contributing to a network of trust among the project participants. The overall objective of the DELTA project was to develop models, tools, methods and a way of thinking (tänkesätt) about the interaction between information systems (IS) and the use of these by people doing systems development. The project was intended to cover new and/or updated theories on 1 Management principles and models 2 Dimensions of complexity 3 Control objects, that is the entities that management manages The project s designers wanted to bring a holistic perspective to the development of complex enterprise and information systems, including aspects such as politics, economics and law. In this way, DELTA should resolve What management principles, theories and models can support the co-ordination of enterprise-wide development and information systems development? What conditions require coordinated developmental activities? What factors make this kind of co-ordination difficult in large organisations? What managerial and intellectual instruments such as theories, models, methods and tools are available to address the difficulties of co-ordination? What architectural framework can support the systematisation of existing knowledge in order to provide sound support to the management of an everdeveloping enterprise? Volvo Parts (VP) and the military high command participated in the project as providers of real systems development situations. VP is a rather new company inside the Volvo Group (4 years), involving its development managers experienced within global logistic systems. Logistics at VP is a complex operation characterised not least by intensive information exchange between many stakeholders (1.000s of suppliers and 10.000s of distribution points towards 100.000s of end-customers). The industrial product families it deals with contain some 100.000s of parts so that VP has to handle both a

long-term service responsibility and complicated supply chains. The parts are also increasing in complexity over time, as they are no longer just hardware but also involve software as well as service arrangements and larger applications. At Volvo Parts the knowledge generated provided some direct input on how to set upprojects with an optimal size, better co-ordination of different project-teams, create new networks both outside the company and inside between different groups. As one participant concluded, the project perhaps did not produce new tools and methods but a lot of intangibles that are difficult to value. It created an awareness inside the company on the human sides of the complexity, issues that usually had been treated as technical matter rather than a social, given that 75 percent of the issues in systems design are soft and not technology related issues. The scale and complexity of advanced military systems is well known, especially in the field of command and control. As at Volvo Parts, tackling such systems involves co-ordinating vast amounts of information but with the added problem that this needs at be at in, or near to, real time. The project is very important to the military because it helps with the much bigger Nätverksbaserade Försvaret (NBF - networkbased military) project, whose aim is to attain information dominance of the battlefield. NBF implies much tougher systems development needs and criteria than in the past. The military now also has a huge historical IS investment, so a major concern is not just development of new systems but change management in relation to old ones. (This is a problem that is common with the civil parts of government.) The defence contribution to DELTA was essentially to provide case studies of military problems and interviews with those involved. Integrating the stakeholders perspective in defining systems to be developed or changes is decisive for success. Both the customer/user and the developer have to have the same common picture of the system and what it does. (Compare the traditional cartoon about what the data processing department actually made when it was asked to hang a used tyre on a rope from a tree, in order to make a child s swing.) The main activities of the project were workshops, and students attending a Masterslevel course, IS/IT planning and management, thus putting the DELTA situation model into practice during the Autumn of 2000. The project also allowed one industrially-financed doctoral student (industridoktorand) to participate from Volvo Parts. A crude estimate is that Volvo was contributing about 500 000 SEK/ year as well as considerable time in-kind. Given the breadth and difficulty of the problems tackled by the project, one can raise questions about whether the amount and level of seniority of effort were adequate to tackle such a big task. NUTEK administration was seen as a negative influence in this case. After the prestudy, which was finished in March 1999, there was a gap of several months before DELTA could get the go-ahead so it was then hard to put participant team back together. The team was also anxious about the receptiveness of the administration to a project, which was 75% concerned with soft factors rather than hard technology. There is a general sentiment among participants that the DELTA-project did not get as far as expected. One main result, however, was that they managed to figure out how to think about the problem of choosing models and tools. Hence, the 52

conclusion of the project steering group of the project was that progress has been made and no one else has got any further. The ways of thinking generated in DELTA influenced work styles, especially among the different users. The project was therefore highly rated by participants. A DELTA2 project has been launched, which does not rely on subsidy but is directly funded by the industrial participants. While DELTA s achievements have been limited, several participants as well as project final reports pointed to the need for more research. The main messages were about the need to do more research about what the problem is, rather than rushing off to invest more money in fixing the wrong problem, and to get unconventional thinking accepted. These were, of course, also among the objectives of the entire KTS programme. 6.2.2 A New Conceptual Framework for Design One of the research groups that had support from the predecessors of KTS was the group around Bengt Lennartsson at the University of Linköping (ULi). The group had been carrying out research on common issues in engineering, product development and design, particularly on soft issues and tacit knowledge (tyst kunskap) in product development. Exhibit 35 illustrates the general engineering problem. Thus, one might say that the overall goal of this particular project inside KTS was to develop management models based on shared understanding. Exhibit 35 Formulation of the General Problem in the Project Science Describe objects in nature Engineering Realise objects that do not exist Design Design object that do not exist Source: Bengt Lennartsson The group organised its research under the KTS programme as part of an internal umbrella programme by the name of NySum (Ny Systemutvecklingsmodell). This is an interesting formalisation of what most academics do when they look for funding, namely: to decide a programme or trajectory of research and to look for funding that allows them to implement this programme. Thus, when NUTEK looks at this New Conceptual Framework project, it sees one among many projects intended to help it reach the KTS programme goals. From the ULi perspective, the project represents one part of the funding portfolio for its NySum programme which may have quite different objectives to those of NUTEK. The New Conceptual Framework project involved Ericsson Utvecklings AB and Whirlpool Sverige AB as industrial partners. It contained PhD-projects, actionresearch and projects that contained both elements. 53

The long-term goal of the project has been to develop methodologies and support systems for the industrial development of complex, adequate, and reliable systems. The focus is on how to establish a shared understanding of basic concepts and system architecture and behaviour among users and developers early in the project. In this way, the developers both understand what the user needs and work as a team to deliver it. An overall hypothesis is that this understanding can be supported, but never replaced, by methodology and support systems. The stages in the project are therefore 1 Packaging findings (in system architecture, software reuse, team learning, system modelling and behaviour visualisation/animation) from previous and on-going projects. This group sees this stage as involving knowledge transfer from university to industry 2 Identifying needs in the real development processes not covered by existing/known methods and tools, especially using case studies. The group sees this stage as a knowledge transfer from industry to the university 3 Introducing and evaluating new methods and support systems to establish a shared early understanding among the development teams and external parties of what a system should do, why, and how to achieve this. This stage involves action research across university and industry in order to develop a new, shared understanding The New Conceptual Framework project has several sub-projects, two of which we describe here. One of the sub-projects is a PhD project that combines the general problem with a concrete application at Ericsson. The other is new a product development process at Whirlpool Microwave Oven that was inspired by the results produced inside the overall project. 6.2.2.1 An interaction-based approach for structuring co-ordination activities at Ericsson Coordinating development activities is critical when developing large software systems, because of the huge combinatorial complexity involved. For example, the software of a single node in the 3 rd generation mobile telecommunication network (3G) may contain millions of lines of program code, which are developed in several increments by designers all over the world 25. These increments are in general dependent on each other and must be integrated and tested in a certain order. System requirements often change during the development. This makes the co-ordination task even more complicated and introduces new problems, because it is very hard to track and predict all the consequences of change. Such development tasks cannot merely be regarded as simply technical matters. Cognitive and social aspects have to be considered as well. Complex systems design is done within some sort of organisational and technical framework. The focus of the research has been to identify the effects of such a framework on the way the project is coordinated. But co-ordination is not the only activity needed in the development of complex systems. In the literature it is 25 Taxén, L, An Interaction-based Approach for studying Co-ordination Activities. Ericsson AB/Department of Computer Science, University of Linköping 54

customary to separate between productive and managerial activities and to argue that both are important. But in industrial practice, normally only one of the constituents is in focus at a time. Therefore the theoretical perspective in the project was to emphasise intersubjectivity and shared understanding across productive and managerial activities. The project focused on understanding the relationship between individuals cognitive understanding of their work and the rest of the development project, and to test the model at Ericsson. Using this framework, a development model has been implemented at Ericsson since the early 1990s. It has been through a number of changes and refinements and was even on the brink of being abolished in 1999 because of serious information system performance difficulties. Currently there are about 30 main projects and subprojects, which use different parts of this framework at Ericsson. The New Conceptual Framework sub-project refined one aspect of the model. The spread and use of the model have been valued highly at Ericsson. Some project managers regard it as essential for the co-ordination of a complex system such as developing the 3 rd generation of mobile systems. The framework is generally regarded as providing advantages in terms of implementation time, cost and functionality. It also contributes positively to the activity of coordinating the development of complex systems. One of its main benefits is to allow continuous improvement in operations, to provide good support for configuration management and engineering change requests, and to manage new requirements in an effective way. Other positive effects have been to increase the level of shared understanding within the development team, the ability to resolve conflicts, increase performance and alter the division of labour in development projects. The research problem is generally regarded as fairly established and recognised. Previously, only hard facts have counted at Ericsson, even when the social aspects of systems development have been the most challenging. The main contribution of the project is therefore to recognise these social aspects of the development process and exploit this knowledge in a concrete industrial application. 6.2.2.2 A new product development process at Whirlpool One of the industrial companies that participated in the umbrella organisation was Whirlpool Microwave Ovens, which has a long-standing relation to Bengt Lennartsson s research group. Roland Ekinge, head of microwave theory and techniques in product development at Whirlpool, saw similar soft problems in his business to those arising at Ericsson 26. Whirlpool faces a rapidly changing environment. The traditional way of managing the product development cycle has been top-down, through planning, action and control. However, an ever more competitive environment demands more rapid and efficient product development. The relevant stock of knowledge is doubling every 5-6 years, the workforce is becoming more mobile and the speed of innovation is becoming faster, with shorter and shorter product cycles. All these changes must be linked to product development changes inside company. Again it is the social aspects of the 26 Ekinge, R, 2001, Socratic Leadership and Three Dimensional Change,, International Engineering Management. 2001. 55

development process that constitutes the level of complexity. Partly this is a function of the large number of people and groups involved in developing and designing a new product in different locations: Whirlpool has product development centres in Sweden, China, Brazil and the US. It has to be seen as a more decentralised change process. The design problem therefore has to be reconceptualised in order to Foster a reflective questioning and critical attitude Foster the ability to manage around the obstacles Be seen as an activity and not a discipline Foster the capability to understand and formulate problems Ekinge implemented the general ideas in the New Conceptual Frameworks project in designing a new product development process at Whirlpool. Some of the main changes that resulted were No more use of product specification documents These were replaced by intensive dialogues to create product definitions and technical trajectories New types of design reviews were introduced, based on shared understanding Development of a formal design methodology Reorganisation of the product development organisation The result of this reorganisation using the knowledge and concepts generated in this KTS-project was thus immediately put to the test at Whirlpool. It tried to change behaviour through a change in thinking. The new process did not use detailed products specification documents. Instead teams were encouraged to work using Product Definition Documents that consisted of a one page animated picture. Thus, the process did not make use of any information system tool and yet it worked. The new product development methodology remains rigorous, and involves both increased formalisation and increased attention to soft aspects of the design itself and of the development process. A new microwave oven by the name of Maximo was the first result of the changed product development process. 6.2.3 Sensor Data Integration and Force Control In contrast to the first two examples, where the human dimension is central, this project is highly technical. It is regarded as complex only in the sense that it involves interaction between a robot and the external world with unpredictable feedback. Traditionally, robots make sequences of pre-programmed moves and do not use sensor feedback to change what they are doing. This is why they often are contained inside cages, so that people cannot accidentally get in the way, because the robot has no way to understand someone is there. This problem was graphically illustrated at Kawasaki some 20 years ago, when a heavy robot crushed a worker to death who was unlucky enough to be in the wrong place at the wrong time. Even now, while it is possible to program a small robot to pick up an egg, it can only pick up that particular egg: it will drop a smaller one and crush a larger one, because it cannot feel what it is doing and adjust its grip. 56

One of the central problems this project tackled was how to use sensors, so that robots can feel in this way. This required close co-operation with ABB, which provided the robotic equipment. The university and ABB had to work together on a new generation of operating system for the robot, which could process feedback from force sensors in real time in our example, therefore, letting the robot feel the egg before it crushes it. The project integrated not only pressure sensors but also cameras. A key example problem was to train the robot to catch a ball. ABB has held for a significant time more than half the world market in heavy robot equipment with a particularly strong market presence in car manufacturing. It has an installed base of over 100 000 robots. Robot control systems and the control systems of other manufacturing equipment are traditionally closed. This has hampered system integration of manipulators, sensors and other equipment and such system integration as has been achieved has often been done at an unsuitably high hierarchical level. With more open systems, robots could better be integrated into production and ABB could, as a result, potentially increase its robot sales. ABB s interest in this project has therefore been to show how to organise open robot control systems and to verify these ideas experimentally. Some of the research issues relevant to ABB are Control methods for adaptive path tracking Developing methods for object identification, object localisation and detection of obstacles Developing programming and control of manufacturing processes (such as arc welding) for industrial use The KTS project that we studied looked at Sensor-Data Integration and Force Control for Task Level Programming. The specific project builds on several years of cooperation between ABB and the University at Lund, as well with other Swedish departments in robotic research. One direction of research represents an enhanced use of sensor information in robotics. Another focus is a task-oriented robot programming method and a discussion of the associated control system. Benefits of creating a new programming environment include giving reusability, maintainability and reliability to the robot program code, all key factors in efficient programming. The outputs of the project include new knowledge about new methods and new applications. The long-standing co-operation between various departments and ABB robotics has created a tight robotics community. Several of the researchers have spent some time in research and production facilities in Västerrås. Thus, ABB itself contributed significantly to the project. A crude estimate is about 5 persons who contributed actively. One important output of the project was improved confidence from key customers that ABB can deliver complex robot system for advanced applications as assembly and machining. This has made it possible sell robots to customers, who would otherwise hesitate about the capability of ABB to deliver these products in the future. 57

The new product is planned to ship to customers in 2003. It is a robot system for the assembly of automobile gear boxes, using force sensors and impedance control of robot motion. There is now a demonstration version of the product, which has been shown to Ford. The project is a traditional case of close co-operation between research and industrial R&D that has developed cumulatively over several years, which has resulted in a close network between industrialists and researchers. The project addressed important technical problems in a rather mature technological area that in turn created researchable questions in academia. It did not aim to tackle either the human dimension or questions of generic complexity. But, in its own terms, it must be seen as highly successful. The co-operation with ABB continues. One follow-one project, 100% funded by ABB, involves the university in helping with some development issues. Other, more research-oriented projects are expected to follow. 6.2.4 Sensor Fusion for Autonomous Robots: the Electrolux Autonomous Vacuum Cleaner The consumer appliances industry has become increasingly concentrated into a few huge companies in recent years. The Electrolux company has been one of the survivors, during the 1990s buying Zanussi, White AG and Thorn appliance interests. This has allowed the company to build further scale and rationalise production. In the course of this transition, Electrolux has become more of a trans-national than a Swedish company. Its headquarters is now in Brussels. Its Swedish R&D activities have largely been centralised to Germany and Italy. Twenty years ago Electrolux was routinely involved in national technology programmes and was seen as playing an important part in diffusing new technologies (such as robotics) in Sweden. Now, however, it behaves now much more like a foreign multinational and its perspective on Sweden as a location is much more that of a foreign direct investor. Technical change and learning mean that mass production products like vacuum cleaners fall in cost and price over time. Producers struggle constantly to differentiate their products, in order to increase their market share, and to add new higher value products to counter the effects of constantly falling prices on their revenues. Robotic appliances have been the stuff of dreams and television for many years. Electrolux has been involved with robot technology since the start of the 1980s and for a time, it made robots for its own use. At about the same time, it entered a joint venture with Hugh Engelberger the inventor and entrepreneur behind the Unimation company that originally led the field in industrial robotics in order to work on robot vacuum cleaners. The idea was to start with highly standardised environments, like hotel rooms, minimising the need for the robot to learn about its environment, but available technology was not able to cope. Ten years later, with electronics technology much cheaper and more capable, a robot vacuum cleaner was one of a handful of projects assigned to a New Products Division, comprising 5 people and answering directly to the Managing Director. From the start, the Division worked by using many sub-contractors. They tried a range of strategies to design a vacuuming robot, which needed in terms of the technical challenges involved to be a combination of a vacuum cleaner, a car and a cruise missile. The product would need advances in navigation technology, battery technology, obstacle detection and strategies to avoid getting stuck. In 1991, 58

Electrolux commissioned several projects from Chalmers Industriteknik, linking them with defence contractors and small design consultancies. Soon they had a working prototype, but it was hard to sell internally, as there was clearly much more work to do before it could be turned into a manufacturable product. By 1994, a simpler prototype was produced, which looked much more like a finished product. The new product team persuaded the BBC s Tomorrow s World popular science programme to do a feature on the machine, which then gained credibility inside and outside the company. This prototype provided the basis for the autonomous vacuum cleaner. Before it could go into production, however, a complementary innovation was needed. The cleaner uses a random cleaning strategy, moving around the room and avoiding obstacles, but relying for its effectiveness on cleaning for a much longer time than a human being would spend. The life of the motor has therefore to be three times as long as that for a manual vacuum cleaner, so it was necessary to work with a sub-contractor to develop a new brushless electric motor and with another to devise an extended-life battery. The random cleaning strategy means the vacuum may clean the same spot several times, while leaving others uncleaned. Over several cleanings, the probability that the whole floor area has been covered is high, but the robot works best in combination with an occasional cleaning session by a human being. The robot is sold for several times the price of a conventional vacuum cleaner and has been a success in creating a high-price market segment in several countries. Cheaper competing models are now beginning to appear from US manufacturers, increasing the pressure to make a second generation of robot cleaner. In 1996, KTH set up a Centre for Autonomous Systems, whose focus is on systems for the house a research agenda that is very demanding because it tries to take autonomous systems out of the highly structured environments such as production systems, in which they otherwise operate. The project leader of the Electrolux autonomous vacuum cleaner project was one of several industry people recruited to the board of the Centre, and after some time he invited to the group to help improve the way the cleaner navigates. This involved a research question that appeared very interesting: namely, how the cleaner could generate a map of its surroundings and locate itself on the map while still making the map. Obtaining KTS funding made it easier for Electrolux to be involved in the project, which involved a new and unproven research group and which was not in the mainstream of product development, but which could produce outputs useful in the next generation of autonomous cleaner. The project produced two patents, funded 80% of a PhD student and led to several academic publications. According to contract, the intellectual property rights belong to Electrolux, but academic inventors receive a bonus for any ideas that are patented. It is likely that ideas from the research will be used in designing the second generation cleaning robot. The project has been a technical success, and a success in terms of KTH s research and education objectives. Electrolux has obtained some new knowledge and ideas, which can later be applied in development, and established a working relationship with CAS that will allow it to work with the Centre in future. Whether this relationship is exploited depends, in part, on future Electrolux decisions about product development and the location of R&D. 59

6.2.5 Horizontal Project: Configuration Management This and the next case are very small, and can be characterised as classical synthesis work aimed at diffusion to industrial users. In KTS terminology, they are horizontal projects, aiming to generalise results from the more applications focused mainstream projects. In practice, however, they do not only rely on KTS programme work but incorporate important international developments, so they are more state of the art reviews than a strict synthesis of KTS outputs (which would be less complete and less interesting). One of the research groups that received funding from KTS was the Software Management Group at LTH under the heading of Boris Magnusson. The particular case chosen on configuration management was sponsored by KTS and the Association of Swedish Engineering Industries (VI). The project also included participants from leading Swedish companies. Configuration management is especially particularly important in large development teams that are geographically distributed, nationally or world wide 27. It involves supporting collaboration by managing the configuration of the project: who is doing what, what is the status of their tasks and which is the current or master version of any given module. Distributed configuration management has increased its importance thanks to the use of Internet. The emphasis in the report is on support for fine-grained integrated configuration management with support for development and collaborative awareness. The research is centred around a number of core areas within software development support, with central themes of integrated environments, object-oriented languages (in the tradition of Simula, BETA, and Java), and embedded systems such as industrial robots and mobile phones. The research method is focused on experimental implementation and development of theory that is of practical relevance. Two reports were produced. The first one dealt with the question of Distributed Development and Configuration Management. The second tackled Product Data Management and Software Configuration Management. Both issues are central for the Swedish engineering industry. The synthesis reports reviewed the theoretical issues involved as well as describing different case studies of companies using these software tools. There was a simple and obvious purpose: namely, making and disseminating a synthesis of knowledge in the area and monitoring trends both nationally and internationally. Particularly important was the objective to provide smaller companies with knowledge of how far they can use the tools they have today. The two reports were presented at conferences organised by VI and distributed to a large audience among the member companies of the engineering association. 27 Asklund, U et al, Product Data Management and Software Configuration Management. The Association of Swedish Engineering Industries, and Asklund, U, 2001, Distribuerad utveckling och configuration management, The Association of Swedish Engineering Industries, 2001 60

6.2.6 Horizontal Project: Systems Engineering of Complex Systems One of the horizontal projects initiated late in the programme involved establishing a database of project characteristics and using the emerging ISO 15288 standard as a way to structure thinking about extracting meta-lessons from the programme, which stretch across applications. The standard has been promoted by the International Council on Systems Engineering (INCOSE), and the report from Tom Strandberg and Bud Lawson lays out key systems engineering principles in a way that is simple and easy to understand. Systems engineering is an interdisciplinary collaborative approach to derive, evolve and verify a life cycle balanced system solution that satisfies customer expectations and meets public acceptability. (IEEE 1220) Complex systems are a natural field of application for systems engineering, which therefore involves management approaches and processes for designing such systems. A minority of the KTS projects try to use various systems engineering approaches. While the report provides a useful general introduction to systems engineering, the lack of commonality among the projects and the low degree to which they had emergent aspects contributing to improved understanding of general complexity and systems design issues meant the project could not go far beyond stating general principles. 6.2.7 Conclusion The seven cases described illustrate some of the mechanisms used in the programme to reach the overall objective set up in the KTS-programme. As such, they reflect some of the strengths and weaknesses found in the evaluation, but they are not representative. The first two cases, for example, strongly emphasise the importance of the human dimension something that was not the case in many of the KTS projects. The comparatively limited achievements of DELTA combined with the industrial determination to carry on regardless remind us not only how important but also how difficult it is to achieve the needed interdisciplinarity especially across hard and soft disciplines. The cases do seem to conform well with the overall conclusion of the evaluation that the KTS programme raised capabilities among participants, developed new techniques and led to important benefits for companies involved in the projects. The cases also illustrate the deep involvement of industry in the research agenda, the execution of projects and in the transfer of new knowledge into practice, where possible. Our participant questionnaire showed that the projects overall were seen as core business and the cases do seem to confirm this. The two cases at Ericsson and Whirpool Microwave Oven and the DELTA project highlight one more feature of complex systems encountered in the evaluation. These systems seem to be emergent in the sense that significant and unexpected surprises are common, and these can lead to redesign chain reactions affecting multiple modules of a design and the organisation responsible for them. Interdependence and the need for systems understanding mean that modularising knowledge, organisation and design can a poor strategy. Rather, as the cases of Ericsson and Whirlpool show, 61

knowledge needs to be managed in an integrated way. They also indicate that globalisation is an important source of complexity, to which approaches to complex technical systems design need to respond. Certain of the cases have a rather tenuous connection to complexity as it was discussed in the programme. More generally, the cases also point to a weakness identified in the overall evaluation. While some of the participants may have been capable of making progress towards the needed generic systems development theory, the programme lacked an adequate mechanism for encouraging them to do so and consequently the individual projects described above had limited incentives to contribute to this. The horizontal projects described, which were originally conceived as synthesising results from the programme, have in practice had to become state of the art reviews 28. 28 It is worth noting that this is not true of all the horizontal projects. For example, one such project does synthesise software engineering work done under the programme by seven university groups 62

7 Complex Technical Systems in an Agency for Innovation Systems An overall conclusion of this evaluation is that the KTS programme was interesting, useful and well executed. It was additional, it raised capabilities, developed new knowledge and techniques and led to important benefits for participants. Our evaluation of the programme in its context and in its time is positive. However, we do evaluation not only in order to make judgements about whether programmes reach useful results or whether they are well run. Evaluation is an important way to learn, and KTS offers some interesting and, we think, useful lessons for the future. To be useful, we need to consider these lessons using current theory, since VINNOVA as an agency has been conceptualised on the basis of innovation theory that was only just beginning to be formed in the early 1990s, when the KTS programme was conceived. Also, in trying to make practical as well as intellectual progress in tackling generic complexity issues in industrial design practice, KTS bit off more than it could chew. This is not a normal type of conclusion in an evaluation of NUTEK or VINNOVA programmes, but neither is KTS a normal programme. KTS tried, as the test pilots say. to push the envelope : that is, to see how far it could go beyond what is normal in this kind of technology programme. At the very least, we can conclude that KTS tackled some of the more difficult aspects of modern technological development. These difficult aspects include the desire to integrate human beings and human factors into thinking about complex technical systems and the programmes explicit aim to be interdisciplinary. Like other NUTEK/VINNOVA programmes, KTS is a Mode 2 programme, involving industry, university and other R&D workers. This type of research is becoming increasingly important as new technologies evolve at the boundaries between disciplines and as the capacity to do R&D increases in all parts of the economy. In some cases, simultaneous R&D and more fundamental work may be helpful in supporting and creating opportunities for innovation. Researchers may also move back and forth between the two Modes, according to need. Knowledge needs are constantly changing. Programmes could help create communities of Mode 2 practice, in order to spread generic knowledge and also to help achieve links with longer term work that can break bottlenecks in the innovation process, such as those encountered in KTS. A key bottleneck was the lack of an adequate generic systems theory, or a hypotheses within the KTS programme about what such a theory might look like. The stress laid in the programme on industrial participation tended to squeeze out the more fundamental work needed, and the KTS programme lacked incentives and instruments that could overcome this market failure. This need for more fundamental knowledge mean that KTS needed a strategic research component in addition to conducting co-financed R&D. Its ambition better to incorporate human factors is also easier to address at the level of individual techniques than technologies, but work may be needed on the knowledge represented by the technologies in order to achieve desirable aims. Again, this may require more fundamental work than it makes sense for industry to fund. Different project structures, for example involving more end 63

users, could also help improve the ability of the research funded to integrate human factors. Other configurations (such as larger, multi-department projects) might have been useful for increasing the amount of interdisciplinary work in the programme. Not least, the very technical focus of the programme has tended to push not only these aspects down the agenda. In an innovation systems agency context, it would make sense to take account of the different ways that low volume complex product systems and high-volume complex products are produced, because these appear to require different hard and soft technological approaches. 7.1 Knowledge Production and Innovation Just after the first KTS programme was launched, Michael Gibbons and colleagues 29 brought together a lot of recent thinking about how research is done in a distinction between two modes of knowledge production (Exhibit 36). Theirs is a simplification 30 of a complex reality, but one that gives us some useful concepts for tackling policy and research administration. Mode 1 is disciplinary science, and can often be basic science, though applied science can be done in Mode 1, too. Its logic comes from its internal organisation and control mechanisms. Its institutions tend to be centralised and stable. In terms of education, Mode 1 tends to provide basic training and a disciplinary entry ticket (such as a PhD) for people to qualify as credible researchers in either Mode. However, Mode 1 is not the same as basic science. Research that is in some sense fundamental or long-term can be done in either Mode. Exhibit 36 Mode 1 and Mode 2 Knowledge Production Mode 1 Mode 2 Problems set and solved in the context of the (academic) concerns of the research community Disciplinary Homogeneous Hierarchical, tending to preserve existing forms of organisation Internal quality control Problems set and solved in the context of application Transdisciplinary Heterogeneous Heterarchical, involving more transient forms of organisation Quality control is more socially accountable Mode 2 includes not only the practice of applied science in universities and other research institutions but also the generation of research-based knowledge elsewhere in society. Mode 2 work tends to be transient. It forms and re-forms around applications problems. Calling on different disciplines and locations at different times, it is hard to centralise. Since Mode 2 work is performed in an applied, social context, it is normally subject to social and economic evaluation, and not solely to 29 30 Michael Gibbons, Camilla Limoges, Helga Nowotny, Schwartzman, S., Scott P. and Trow, M., The New Production of Knowledge, London: Sage, 1994 Gibbons and colleagues also get their history wrong, claiming that Mode 2 is new. In fact, it is Mode 1 that is historically new, while Mode 2 is the traditional form of science, as practised for many hundreds of years 64

traditional quality reviews by scientific peers. To the occasional irritation of those used to the Mode 1 tradition, this means that relatively frequent evaluation in part by non-scientists is normal in Mode 2 work, and has become part of the new social contract between scientific researchers and society. 31 The sharp distinction between Mode 1 and 2 can make it seem as if they are alternatives. Many researchers, however, do both, 32 so they take closely related research problems to different research agencies to ask for funding. The current National Innovation Systems approach to economic development and growth is based round the idea that many interconnected and systemic factors are important, but has not yet got very far in describing how the interconnections need to operate in order to make the system work. Innovation drives growth because there is constant competition to improve. Unlike in the neo-classical economic models, where technological change is seen as external to the economic system so that equilibria tend to develop, the innovation systems approach sees the constant evolution of technology as internal to the system. Because there are no optima, each change in technology creates a new set of economic and technical opportunities, under which economic actors compete to create advantage, so that innovation constantly triggers new innovation (Exhibit 37). Exhibit 37 Evolutionary View of Innovation in Innovation Systems Other producers and users make complementary and competitive innovations Creativity based on old and new knowledge Selection environment Altered pattern of economic activity New knowledge based opportunities Framework conditions Source: While the diagram is our own, it was inspired by JS Metcalfe, Co-Evolution of Systems of Innovation, paper presented at the Volkswagen Foundation Conference, Prospects and Challenges for Research and Innovation, Berlin, 8-9 June 2000, CRIC, Manchester University 2000 31 32 Ben Martin, Ammon Salter et al, The Relationship Between Publicly Funded Basic Research and Economic Performance, report to HM Treasury, Brighton: Science Policy Research Unit, 1996 We explored an example of this in practice in Erik Arnold, Catherine Whitelegge, Feroz Ghani, Nigel Harris and Tore Nordin, Evaluation of the Railway Research Group at KTH, Stockholm: KFB, 2000 65

A key aspect of the innovation systems approach is therefore to understand that innovation is not only about technical change but its interplay with other factors. It is not only about new knowledge but also about the re-use of old knowledge. And it often involves other things than those we think of as technical. In recent years, new technologies have nonetheless enabled rapid economic development. Many are ones that we think of as hyphen fields 33 such as optoelectronics, bio-engineering, nano-technology and so on, where an explicit or implicit hyphen shows that the field works across the boundaries of more traditional disciplines. These new growth fields are essentially Mode 2 activities. Research funding for them would tend to be ruled out by the internal rules of disciplinary Mode 1 science. The massive use of science and technology in industry and the increase in industry s links with external research which has brought the share of Gross Domestic Product (GDP) devoted to R&D up to almost 4% in extreme cases like Sweden, involves mostly Mode 2 activity. This represents an important challenge to research policy and funding bodies. Especially in fields like biology and Information and Communications Technology (ICT), fundamental and more short-term research problems are increasingly being tackled in parallel. Over the last 20 years, engineers in industries that make complex products have begun to design different parts of these products at the same time socalled concurrent engineering. As they need to tackle more fundamental problems at the same time, we can begin to talk about concurrent engineering and science. Even relatively fundamental research is therefore pulled into the innovation logic described in Exhibit 37. We will argue in the next section that KTS ran into territory where fundamental theory was lacking, creating a need to do concurrent science and engineering. The KTS programme works very clearly in Mode 2, as do most of the NUTEK/VINNOVA programmes. A quite explicit objective is to pull the university participants out of Mode 1 and into Mode 2. Of course, many of them are happy to operate in Mode 2 and regard this as normal, but the institutional structures and academic reward systems within which they live are generally more oriented to Mode 1. The KTS programme, and especially the difficulties the second KTS programme encountered in trying to span its large scope, also highlights a problem of knowledge accumulation in Mode 2. It is clear, for example, that the defence-related simulation projects and the robotics projects live in different knowledge communities or communities of practice. They both to some extent connect to Mode 1 knowledge, for example that which is published in the disciplinary journals. But the R&D practitioners to whom they relate meet and publish in different places and barely touch each other. We would expect something like KTS to be a potentially important vector of new understandings among these different communities of practice. This should work in part through mechanisms like the programme conferences and through the creation and transmission of more generic knowledge among the different communities in the programme. 33 Following Frieder Meyer-Kramer 66

7.2 The Theoretical Problem Earlier, we suggested that KTS ambition to develop generic complex systems design knowledge was in part defeated by the fact that this was not in the short-term economic interest of the companies involved. An aspect of traditional market failure concerning research came into play: namely, that companies have weak incentives to fund more fundamental research, because this is hard to appropriate. Most of the benefits of an industrially co-funded KTS project that actually generated such generic knowledge would not flow to the industrial funder but to others. In economic jargon, such work has high externalities, so it is more rational for the state (on behalf of society) to invest in it than it is for individual companies. However, the more generic or fundamental questions that KTS tried to answer also ran up against another serious problem. The theoretical basis was weak. One of the key intellectual roots of KTS is in the systems thinking that emerged in the 1940s and 1950s, stimulated by the potential of emerging computer technology that had been spectacularly demonstrated in theory and practice (such as code breaking) by Turing and others. Norbert Wiener 34 (Cybernetics) and von Bertalanffy 35 (the idea of a General Systems Theory) were leading figures in this movement. As Ingelstam 36 points out, they have important ideas in common. One is the principle of nonreductionism: the idea that the behaviour of systems cannot be explained only by reference to their components, but that it is also determined by phenomena at higher levels. Another is the idea of interdisciplinarity or borrowing tools among disciplines. The third is that they envisage a core of systems theory emerging as a new and viable discipline or meta-discipline in its own right. Here we have a clear intellectual link to the structure of KTS, which aimed at (1) working in complex systems domains to improve understanding and provide bases for developing more general methods, (2) developing generic methods in systems design, and (3) developing tools to implement such methods. The key contribution of KTS was intended to be to the kind of new or meta-discipline envisaged in the earlier work on systems theory. However, in practice, rather than becoming a basis for a new science of systems, the work of the general systems pioneers has been absorbed into a large number of disciplines, which now take a more systemic approach to their fields. Thus, in control (reglerteknik), the approach has become more systemic, control has worked through various adaptive strategies up to and including the use of fuzzy logic, but a deeper integration with computer science is yet to happen. Software engineering has evolved a succession of techniques to achieve and manage the development of software. A number of development methods and tools have emerged, which bring together technical and managerial aspects of software development, but this fusion has not been generalised into other design domains. The long tradition of automating aspects of hardware design Computer-Aided Design/Computer-Aided Manufacture, Computer-Aided Engineering, product modelling and so on continues to make progress, but without tackling the integration with management in a major way, and 34 35 36 Norbert Wiener, Cybernetics Control and Communications in the Animal and the Machine. New York: John Wiley, 1948 Ludwig von Bertalanffy, General Systems Theory, New York: George Braziller, 1068 Lars Ingelstam, System att tänka över samhälle och teknik, Eskilstuna: Statens Energimyndighet, 2002 67

with little spill-over to more generic systems design issues. Modelling and simulation techniques also progress, and are being used in an increasing number of design processes and stages of design, reducing the need for physical modelling and prototyping without obviously contributing a great deal to a meta-level of understanding about systems and design. This tendency for general systems theory to disappear is important, because it means there is no obvious body of existing knowledge from which KTS could start work in order to realise its intellectual ambition to advance knowledge and practice about generic systems design. A problem definition, which (1) more directly tackled generic systems design issues, though still in combination with work on applications, and (2) involved more space for the human element and the innovation process, might have been more scientifically useful than the one KTS actually chose. But it would also have involved a higher level of scientific ambition than the KTS funding and governance structures could provide. We can think of this needed, more generic systems theory as being in a similar position to thermodynamics 200 years ago, which spent a long time catching up with industrial practice. It first had to explain why the new steam and atmospheric engines worked at all before it could, through theoretical refinement, begin to explain how such engines could better be designed. The development of thermodynamics was an early case of concurrent science, but in rather slow motion. If it is necessary today to force the pace, then the kind of bottom-up, researcher-driven research funded by the Swedish Research Council is unlikely to serve. More focused funding will be needed, and some of this funding will need to have a smaller industrial component than was allowed under the KTS rules. 7.3 The Human and Social Dimension of Technology In contrast to the traditional view of technology as socially neutral, it was important for the success of KTS that social and economic needs should be incorporated within the techniques developed. This was therefore a quite radical agenda. Over the past thirty or so years, a considerable amount of literature has built up which attacks the idea that science and technology are inherently neutral. As recently as 1980, radical critiques of science focused on its use as a means to oppress social groups, for example women 37. But from about that time, new critiques of technique and technology began to appear in the Marxist labour process tradition 38 and the feminist 39 literature, which argued that the very shape of technology and technique reflected social interests. Eventually, also writers in the sociology of science 40 started making similar claims for science, as well as technology. 37 38 39 40 Brighton Women and Science Group, Alice through the Microscope: The Power of Science over Women s Lives, London: Virago, 1980 The key text in this tradition is Harry Braverman, Labour and Monopoly Capital: The Degradation of Work in the Twentieth Century, New York: Monthly Review Press, 1974 Where we have ourselves contributed, see for example Wendy Faulkner and Erik Arnold, Smothered by Invention: Technology in Women s Lives, London: Pluto Press, 1986 Bruno Latour, Science in Action: How to Follow Scientists and Engineers through Society, Harvard University Press, 1987 68

In parallel with these radical critiques, mainstream studies of innovation have, since the 1960s, increasingly emphasised the role of competent users in innovation 41. Some of the micro studies of capital goods, such as Shimshoni s, focus on the way user needs are incorporated into design characteristics of particular products (techniques). The broader economics literatures on product cycles 42 and evolutionary economics 43 focus more on the role of user influence over the shape of techniques in consumer goods, through competition among designs. Meanwhile, a belief in the need for enduser involvement in technology design and transfer processes has become orthodox in both writing and practice in technology transfer and development. We hesitate to take this line of argument too far, because we suspect there are real technical limits, as well as scientific limits, to society s ability to impose its will on the design of techniques. However, not least through activities like KTS, society does clearly try to shape technological as well as technical development. In the case of KTS, this is done for good, economic reasons. To understand these limits, we need to make some distinctions. We normally make a sharp distinction between scientific and technological knowledge. Science, especially natural science involves discovery uncovering, in a sense, the rules or laws God used in putting together the universe. But a science like cybernetics involves a different kind of knowledge: not of regularities in nature but in the properties of things that we make, what Herb Simon called 44 a science of the artificial. We also need a distinction between technology and technique. Technology is the scientific study of the practical or industrial arts that is, rigorously organised knowledge about how to do or make things. This certainly relies on the sciences of the artificial and needs knowledge of the natural sciences. It also includes elements of skill and experience or art, which the Royal Institute of Technology in Stockholm so neatly encapsulates in its motto vetenskap och konst. Technique is an instance of technologies being used in a specific application, such as particular machine. Because techniques tackle problems, their creation often involves some kind of interdisciplinarity in order to exploit different kinds of technology and they are tested in interaction with users. In Exhibit 38, we sketch the scope and mechanisms of such influence against four rather arbitrarily separated types of scientific and technological activity. Agenda setting is a key mechanism, which appears to be important at all stages, and to affect science as well as the other activities. Where you look affects what you find, and where you look is strongly affected by what your interests are. As we move to the 41 cp Jakob Schmookler, Technological change and economic theory, American Economic Review, 55, 1965, pp 335-341; Eric von Hippel, The Dominant Role of Users in the Scientific Instrument Innovation Process, Research Policy 5, no. 3 (July 1976) pp 212-39; Roy Rothwell, Successful Industrial Innovation: Critical Factors for the 1990s, R&D Management,:3, p 221-239, 1992; Roy Rothwell et al, SAPPHO Updated - Project SAPPHO, Phase II, Research Policy, 1974, 3, pp258-291 42 William Abernathy and James Utterback, A dynamic model of process and product innovation, Omega, 3, (6), 1975, 639-656. 43 Richard R Nelson and Sydney Winter, An Evolutionary Theory of Economic Change, Harvard University Press: 1982 44 Herbert A Simon,, The Sciences of the Artificial, Third edition, Cambridge MA and London: The MIT Press, 1996 69

right in the diagram, the activities become increasingly use-oriented and the role of design becomes more important. Design is the process that deliberately connects technical characteristics with social needs. Exhibit 38 How Socio-economic Influences Affect Science and Technology Natural Sciences Sciences of the Artificial Technology Technique Social influence Agenda setting Design We can at the least conclude that stakeholders have an important influence at all stages. Mechanisms differ. In order to influence the direction of technical change, programmatic interventions need not only to address the quantity of technological and technical effort but also the type of such effort. A good example is NUTEK s DUP programme during the 1980s, which aimed not merely to improve process industry control technology in a general sense but also to enrich and make good use of the skills of the human operators in the systems. This contrasted with the mainstream of development, which tends to involve increasing automation, reduced labour inputs and deskilling. DUP aimed to influence the technological trajectory of control by building particular techniques in new ways. Similarly, NUTEK s MTO programme aimed to reintegrate people, techniques and organisation. KTS conforms with this tradition of programmes, which aim socially to influence technological trajectories as well as techniques. Such programmes have a quarter-century history in the Nordic area, stretching back to programmes such as DEMOS and UTOPIA in the 1980s. However, a key novelty of the 1990s programmes (DUP, MTO, KTS) is that they are no longer oppositional (aiming to strengthen the workers hand against the power of the company). Rather, they aim to increase competitiveness through a more holistic integration of the social and the technical. Because it is about design, KTS operates at the point where it is most clearly possible to influence the direction of technical change. In the abstract, we would therefore expect it to be well positioned to integrate the human dimension with complex systems technology, and there have been some successes within the programme. Yet, despite the concern of the programme boards with this question since the early 1990s, there is general agreement that more still needs to be done. Only two KTS projects have a central focus on people in relation to complex systems, though a larger number of others (two of them are described among our case studies) involve aspects of human factors 70

Process modeling, operator training simulation and optimization applied to paper board manufacturing Operator interfaces and alarm presentation (one of the 5 projects led by women) Our work does not allow us to give a definitive answer, but it is likely that the following factors have been influential. While a number of projects 45 tackle questions relevant to the process industries, the desired process industry presence in the programme was never secured, so one of the most natural channels for bringing in the system operator perspective was missing. In general, the process industries are users rather than developers of complex technical systems, and this in turn explains their low level of involvement in the KTS work End users more generally are rarely present in KTS projects. Rather, in KTS the users are the system builders, on the assumption that their knowledge includes an understanding of end user needs. Most of the thinking about the human dimension is therefore engineering driven, rather than people- or user-driven Except for a handful of horizontal projects, which aimed retrospectively to extract generic knowledge about systems, KTS was made up wholly of industry cofinanced projects. The programme managers went to unusual lengths to ensure that projects reflected the interests and needs of industry, and our questionnaire indicates that the projects overall were even more core business focused than is normal in advanced technology programmes. An unexpected effect of this heavy industrial focus has been to crowd out exotic research questions such as the human dimension and perhaps even the central issue of generating new knowledge at a meta-level about complexity Following the line of the radical critiques of technology discussed earlier, the technologies ( or knowledges ) tackled by the programme are not well developed as regards the human dimension. In developing new techniques within KTS projects, they therefore tend to shape the projects in ways which reproduce this focus on the technical, as against the human Gender may even be an issue. According to the data we have from VINNOVA, KTS projects were overwhelmingly performed by men. Only eight projects had female participation. Women led five of these. 7.4 Interdisciplinarity In research, we are constantly presented with the idea of interdisciplinarity as a virtue. In one sense, this is trivial. As Weingarten 46 observes, disciplines do not keep up with rapid developments in modern societies. The map of knowledge, in a sense, is always outdated. The discourse on interdisciplinarity is, in effect, a discourse on innovation in knowledge production. This is one of the reasons why interdisciplinarity is a recurring theme in discussions of systems research: as in the KTS programme, there is a desire to create a new field or discipline. Since it operates 45 46 Arne Otteblad, the original KTS programme manager at NUTEK, estimates that this may be the case in as many as 15% of the projects (private communication) Peter Weingarten, Interdisciplinarity: the paradoxical discourse, in Peter Weingarten and Nico Stehr, Practising Interdisciplinarity, University of Toronto Press: 2000 71

with overlapping areas of knowledge that are very important in modern innovation systems and focuses on industrial activity, we would in any case expect the research to be problem-focused and therefore often to cross boundaries between traditional disciplines. KTS projects span not only the traditional knowledge infrastructure of universities and research institutes but their development is also deeply reliant on research in, and experience of, industry and other parts of society outside the knowledge infrastructure. Without the experience of the factory as laboratory, it is difficult to make progress in parts of the complex technical systems field. The composition of the programme, however, suggests rather less academic interdisciplinarity than we might expect. We do not have department-level participation data on all the projects, but the great majority of projects involving universities include only one university or department. Of the projects in the 1999 project catalogue, there were 52 in this category. Four projects linked different departments together within individual universities. Of the 7 projects involving two universities, 3 involved departments working in different disciplines, 2 brought together departments in the same discipline and in 2 cases there were no departmentlevel data. Of course, departments are by no means perfect proxies for disciplines. These data are consistent with use of the factory as laboratory but not with a major interdisciplinary effort. Inspection of the project titles leads us to the same conclusion. We suspect that a key problem is that projects are not big enough. Three quarters of the projects have budgets of less than 5 MSEK which is perhaps a minimum scale for even small interdisciplinary teams to operate across the three-year period of the funding. If we take seriously Weingarten s point and it is by now rather conventional wisdom then VINNOVA will need instruments to allow funding of larger interdisciplinary projects in future programmes. 7.5 Competitiveness and Innovation Systems The KTS programme chose to take an explicitly technical approach to complexity, focusing on the technologies and techniques relevant to the design and production of complex product systems. It aimed to include the human dimension, as far as this was relevant to design, but made a choice to avoid the wider systemic aspects of innovation. In the context of the early 1990s, when KTS was designed, this was perhaps a reasonable decision. A decade later, and especially in the context of an Innovation Systems agency, a different decision might be appropriate. If one takes an innovation systems perspective, complex technical systems often interact with the economy in the form of complex product systems (CoPS). These have massive economic significance for the competitiveness of the West, and for Sweden in particular. The post-war era has seen a steady erosion of Western positions in many mass markets, combined with a growing importance for economies of scale in these markets. It is not obvious that western countries can regain advantages in high-volume, low-cost product markets. Japan has made some progress in complex product systems markets, while the lower wage economies of SE Asia, which have traditionally emulated the Japanese growth path, have weaker positions. 72

However, it would be reasonable to expect their East Asian market positions to strengthen over time, eventually providing a platform for internationalisation. It does not follow that the western countries retain strong positions in CoPS solely because of superior capabilities. If anything, it is likely that this dominance results from the inertia in CoPS markets (which means barriers to entry and exit are high) and the fact that smaller, mass produced products are easier to develop. A major effort has been made at CENTRIM and SPRU, at the two universities in Sussex, over the past few years to understand the characteristics of innovation and management in CoPS. This suggests that CoPS have at least three defining characteristics, which distinguish them from mass produced goods. First, as high cost, capital goods they consist of many interconnected, often customised elements (including control units, sub-systems and components), usually organised in a hierarchical manner and tailored for specific customers and/or markets. Often their sub-systems (e.g. the avionics systems for aircraft) are themselves complex, customised and high cost. Second, they tend to exhibit emergent properties during production, as unpredictable and unexpected events occur during design and systems engineering and integration (Boardman, 1990; Shenhar, 1994). Emerging properties also occur from generation to generation, as small changes in one part of a system's design often call for large alterations in other parts, requiring the addition of more sophisticated control systems and, sometimes, new materials (e.g. in jet engines). Third, because they are high value capital goods, CoPS tend to be produced in projects or in small batches which allow for a high degree of direct user involvement, enabling business users to engage directly into the innovation process, rather than through arms-length market transactions, as is normally the case in commodity goods. 47 Hobday has usefully summarised 48 key differences between complex product and mass product innovations (Exhibit 39). Exhibit 39 Innovations Distinctions Between Complex Product and Mass Product Complex product innovations User-producer driven Business to business Highly flexible, craft based Innovation and diffusion collapsed Innovation paths agreed ex-ante among suppliers, users etc. People-embodied knowledge Mass produced innovations Supplier driven Business to consumer Formalised, codified Innovation and diffusion separate Innovation path mediated by market selection Machinery embodied know-how These analyses suggest, at the very least, that the innovation processes that connect complex technical systems to the market are problematic and need investigation. Other findings from the CoPS work underline the way management, organisation and 47 48 Giovanni Dosi, Luigi Marengo and Mike Hobday, Problem-Solving Behaviours, Organisational Forms and the Complexity of Tasks, paper prepared for the TSER Dynacom project, December 1999, Brighton: SPRU Mike Hobday, Complex Systems vs Mass Production Industries: A New Innovation Research Agenda, CoPS Paper No 5, Brighton: SPRU/CENTRIM, 1996 73

technology are interwoven 49 in CoPS. We can see this in KTS practice in software engineering, where tools may involve a mixture of hard technology and organisational technology, as in development environments and tools. For example, a key strategy for dealing with complexity in CoPS is modularisation. This means that the overall product system has to be decomposed into parts. Finding a workable decomposition is a prerequisite to building the system, but there are normally several alternative ways of decomposing the system, none of which is optimal, and all of which require explicit co-ordination 50. Trade-offs are made here across the dimensions of technology, management and organisation, and these have to be made under uncertainty. Since CoPS are low-volume or one-off products, each new design involves elements of uncertainty. CoPS design is therefore emergent, not only in the sense that CoPS can have characteristics that are not simply additive or predictable via simple rules from the properties of their components. It is also emergent in the sense that significant, nasty surprises are comparatively common, and these can lead to redesign chain reactions 51 affecting multiple modules of a design and the organisations responsible for them. Interdependence and the corresponding need for systems understanding mean that modularising knowledge and organisation as well as designs is a poor strategy. Rather, knowledge needs to be managed in a more integrated way, in the face of modularisation 52. This finding, if correct, has significant implications for how companies should organise, the way they structure design and analysis technologies, and the way they do knowledge management. It is consistent with KTS ambition to find meta-knowledge about complex systems design. It implies that fragmented, disciplinary organisation of education and research in the universities is counter-productive. We conclude that, at the very least, the technical focus of KTS could usefully be extended in a way that treats management and organisation as aspects of technology. However, the scope of CoPS is not the same as that of the KTS programme. Like most people, we feel that technological progress leads to increasing complexity. At the same time, the forces of globalisation, including the cheapening of transport and ICT, and the widespread increase in users ability to adopt and exploit more complex technologies (absorptive capacity) are increasing the scale at which more sophisticated products can be sold. Few countries in the 1950s or 1960s can have possessed as much computing power as the average OECD household today, but it is inconceivable that devices of the complexity of GSM handsets with some millions of transistor-equivalents could economically be produced in low volumes. Exhibit 40 segments this picture of growing complexity and scale. CoPS involve high technical complexity, but their characteristics are negotiated amongst a relatively small number of actors. Producer-user relations are therefore key to success, 49 50 51 52 This is not unique to CoPS. Indeed, in the Nineteenth Century conception, management and engineering were part of the same discipline. Around the 1920s, we began to see the emergence of separate management and engineering literatures a separation which, arguably, has had very negative effects on western industry, cp RH Hayes and WJ Abernathy, Managing our way to economic decline, Harvard Business Review, July-August 1980, pp159-175 Dosi, Marengo and Hobday, Ibid. Paul Nightingale, The Organisation of Knowledge in CoPS Innovation, CoPS working paper No 14, Brighton: SPRU, 1997 Stefan Brusoni and Andrea Prencipe, Modularity in Complex Product Systems: Managing the Knowledge Dimension, CoPS Working Paper No 57, Brighton:P SPRU, 1999 74

suppliers and users co-evolve over time, relationships can be comparatively stable. In contrast, high volume complex products necessarily involve large numbers of users and therefore impersonal buyer-seller relations. Innovations have a bigger trial and error component, with product innovations and producers being eliminated from the market if they fail to please large numbers of buyers. This means that the shapes of the networks that matter to producers should differ between the two types of complex products, and that the management of the innovation process should be different. Tools for understanding user systems and requirements as part of the design process, should be different in CoPS, and so on. Exhibit 40 Technical and Network Complexity High Complex Product Systems (CoPS) Complex, standalone products (mobile phones, televisions, cars ) Technical complexity Low Stranded simplicity : personal services, individual house building, crafts Simple massmarket commodities (raw materials, washing powder, car insurance ) Low Network complexity (No of actors involved) High Scope of KTS programme The trends towards increasing complexity and mass production mean that products and services tend to migrate from the bottom left corner of Exhibit 40 to the other segments. Swedish firms are strongly represented in the top half. Those at the bottom right may have limited interest in terms of complexity. However, this kind of analysis, which tries to situate technical change within the innovation system, offers clues about how to cluster programme activities and design networks of participation that are not as easily available from an analysis that is purely technical. It gives us ways more directly to couple the programme to competitiveness. These opportunities are missed if programmes are defined solely in technical terms. 75

8 Conclusions and Recommendations In this Chapter, we draw conclusions and recommendations about KTS at two levels. We consider KTS performance as a rather routine, shared-cost R&D support programme. At this level the programme was well executed, performed well and produced useful outputs. At another level, the programme was an attempt to advance the state of knowledge that was far more ambitious than those who ran the programme perhaps realised at the time. At this level, the success of the programme was much more limited, but the experience offers several important insights about how to run an innovation systems agency such as VINNOVA. First, it seems that VINNOVA has inherited an inadequate range of funding instruments from its predecessors. The problems KTS tried to tackle could not be solved without a combination of instruments to involve industrial co-operation but also to fund more strategic or fundamental research, for which industry simply will not pay because of the well known market failures associated with longer term research. Second, the structure of the Swedish knowledge infrastructure continues to be an impediment to tackling many of the more important challenges in knowledge production and use, such as interdisciplinarity, scale and the shift in the locus of knowledge production to being primarily outside the universities. There is a continuing need for a change agency here. Third, the governance and incentive systems used need to be tuned to the activities of individual programmes. Those used with KTS tended to lock the programme into the shorter-term concerns of industry and to lock out the work that should have been present to meet its longer term needs. Fourth, VINNOVA needs to counterbalance the excellent tradition of stakeholder involvement in programme design that it has inherited from NUTEK with both economic and technological analysis capabilities, in order to balance short- and longer-term needs. 8.1 Conclusions on KTS as an Advanced Technology Programme The KTS programmes operated in a areas of great importance to the Swedish economy. While the programme managers have not conducted any economic analysis to understand the scope of their potential impact, as study by Heighes 53 found that the somewhat narrower area of Complex Product Systems accounts for 6% of UK GDP. This is at least an order of magnitude indication of the proportion of Swedish manufacturing industry that could be affected by technical improvements in the design of KTS. The original need for the programme was identified, and the programme itself developed, in partnership with key academic and industrial stakeholders, following good Swedish programme design practice. While the start of the second KTS programme was delayed, its management over-stretched and its budget reduced very late in the programme, it nonetheless succeeded in funding projects in a wide range of 53 Tim Heighes, Quantitative Indicators for Complex Product Systems and their Value to the UK Economy, CoPS working paper No 15, Brighton: SPRU, 1997 76

complex technical systems domains. Other evaluations indicate that the scientific quality 54 of the work was broadly good. Most of the project activity induced by the programme was additional: we saw little evidence of free riding. The work of the programme nonetheless proved to be very relevant to industry needs. As in other advanced technology programmes, its participants focused on producing knowledge and intermediate outputs more suitable for use in future technical activities than for being directly commercialised. It focused especially on methods and tools. It produced significant numbers of outputs in the form of higher degrees and commercialisable results. Naturally, industry was more focused than academia on exploitation. As many as one third of the industrial respondents to our questionnaire whose projects were complete indicated that they had already commercialised outputs from the programme. Generally, participants did better at reaching their own project goals than in reaching programme goals. Their assessment of the benefits of participation, as against the costs, was extremely positive. The programme built on past research relationships and experiences to build a large network of interconnected R&D performers, providing an important basis for sharing knowledge. The degree of continuity with the past meant that many co-operations could be strongly based on existing trust relationships. It succeeded in involving SMEs in a technical area that is mainly tackled by large companies. In terms of fulfilling the programme goals, KTS was rather successful in increasing systems development capabilities in the higher education and industrial sectors, and in developing tools and techniques. Its success was more modest in relation to the programme s externalities goals: dissemination of information and transferring capabilities among actors. At the higher level, KTS has tackled a series of difficult problems involving the interfaces between disciplines, industries, people and technologies. The absence of a formal analysis at the start of the programme may be one reason why the difficulty of the key higher-level goal of developing generic complex systems design knowledge came as a surprise. Not only did the programme use an incentive structure in the beginning that did not encourage such projects with high externalities, but there proved to be no strong theoretical base on which to build such generic knowledge. What was needed was not typical pre-competitive collaborative research but something much more fundamental. The management introduced 100%-funded horizontal projects in the middle of the programme, in order to try to improve performance. However, these proved difficult to start because industry and academia alike found them uninteresting and because the programme budget was cut down after only a few such projects had been commissioned. With industry playing the dominant role in deciding what projects to do, the normal (Arrow) market failure relating to research set in, and projects focused on common needs among the participants, rather than producing the more generic outputs, in which the participants had no immediate economic interest 54 Scientific quality is not in scope to this evaluation 77

The expectation that generic lessons about systems design could be learnt depended upon there already being a theoretical basis in place. It turned out that there was no sufficiently powerful theoretical basis for drawing general lessons. KTS tried to bring together what lessons it could by buying horizontal projects, but with no strong base in theory these were rather ad hoc To reach its generic systems development goals, KTS needed to do concurrent science alongside an industrial technology R&D programme, but lacked the means to achieve this. The programme certainly succeeded in doing interdisciplinary work, work involving human factors and the role of people in complex technical systems, but did less of these than members of the steering board wanted, and probably less than was intended at the outset. (There were no quantifiable targets.) Such people-related work tends to be interdisciplinary, and there were not many projects in the programme big enough to work in a highly interdisciplinary way. The programme did achieve a number of links to undergraduate education, though this was not particularly an objective. 8.2 Implications for VINNOVA as an Innovation Systems Agency We think the KTS experience has some important implications for how agendas are set and managed in an innovation systems agency such as VINNOVA. First, the range of funding mechanisms at VINNOVA s disposal is inadequate to handle situations where a combination of industry-oriented and more strategic research is required, in order to build strength in Swedish innovation systems. This means that VINNOVA should acquire money for longer-term or more speculative research and/or that its ability to co-ordinate with other funders should be improved. Second, programmes need to continue to take account of the need for reform and modernisation in the Swedish knowledge infrastructure. In particular, this means using special mechanisms to promote interdisciplinarity and investing in larger projects. Third, technical initiatives such as KTS need to be more clearly understood in their innovation systems context by VINNOVA. This requires better coupling between VINNOVA s considerable analytic resources and the well-established traditions of stakeholder involvement it has inherited from STU and NUTEK. 8.2.1 Strategic Research in an Innovation Agency? Reasonably enough, VINNOVA has put a lot of effort during its short history into date planning its future. This future is determined not only by internal creativity but also by the (rather narrow) role allocated to the agency in the national division of labour. As we read the history, VINNOVA s predecessor agency of two generations back STU 55 had a much more scientific remit. It functioned alongside the research councils as a change agent in areas of technological and industrial opportunity, and had a significant role in funding the universities as well as industry. In this way it was instrumental in underpinning important innovation opportunities for Swedish industry, for example in biotechnology and digital communications. As a network entrepreneur STU was able to couple industrial and research opportunities to see 55 Styrelsen for Teknisk Utveckling, The Swedish National Board for Technological Development 78

the links between often fairly fundamental research and potential industrial activity. This role was important, because the self-governing academic community has a tendency to lock in to existing structures and disciplines, so a change agent that can enable change and growth is an important complement to research councils. STU was merged into NUTEK in 1991. At the same time, a technology research council TFR was set up. This took over some of STU s more scientific funding work and brought it under the control of the research community, which operates Sweden s research councils through a system of election and nomination. After a few years, TFR withered and was absorbed into the mainstream of Sweden s research councils. NUTEK inherited the rest of STU, together with the functions of the national industrial development agency (SIND). NUTEK s functions spanned regional development, start-up company support and innovation support. In practice, much of STU s funding tradition continued within NUTEK s Teknik division during the first half of the 1990s, but major parts of the portfolio were effectively transferred to the Wage Earner Fund foundations as a result of negotiations in 1996. There, these parts of the portfolio were handled increasingly via bottom up allocation processes, in response to project applications. While industrial relevance and involvement are important values at these foundations, much of the money they allocate actually goes to the universities based on ideas produced by the research community. VINNOVA was created in 2001, in effect by combining the remains of NUTEK Teknik with the transport and communications research board (KFB) and parts of the council for working life research (RALF). Superficially, there are strong similarities with STU. However, one of STU s key functions, namely the ability to add strategic resources to academic research in ways likely to be industrially important, has atrophied as a result of successive transfers of responsibility for research, as opposed to industrial innovation, to other organisations. This has produces a cleaner division of labour between VINNOVA and the research councils, but at the price of making it much more difficult to link together different types of knowledge generating activity. KTS perfectly illustrates this, via its inability to make the jump from domain-specific to generic knowledge about designing complex technical systems. As we see it, the last dozen years of continuous struggle and reorganisation in the Swedish R&D funding system has produced a tragic irony. Sweden has embedded the world s first innovation systems agency in an institutional division of labour and system of governance that could only function if innovation systems theory were wrong and the old, intellectually discredited linear model of innovation were right. While other countries have been devising ways to join up the strategies of different parts of their R&D funding systems, 56 Sweden has been going in the opposite direction, apparently convinced that she is the only one marching in step. During the KTS programme, NUTEK/VINNOVA was already suffering from this systemic inability to develop joined up strategies among research and innovation funders. It had no coordinating links to any wider process of setting strategic research 56 Erik Arnold and Patries Boekholt, Research and Innovation Governance in Eight Countries: A Meta-Analysis of Work Funded by EZ (Netherlands) and RCN (Norway), Brighton: Technopolis: 2003; downloadable from www.technopolis-group.com 79

agendas, such as those being developed between TEKES and the Academy of Finland in the same period or those now increasingly being sought in R&D governance internationally. Nor did it have its own funds, with which to finance the necessary strategic research. In short, the institutional context was not friendly to the achievement of KTS higher level and arguably most important goals. 8.2.2 The Knowledge Infrastructure The industrial agenda of KTS is very badly served by the structure of the Swedish knowledge infrastructure, whose weaknesses are as well known as they are often ignored in policy. Indeed, the fragmentation of the funding system and the funders obsession with PhD education during the 1990s have, if anything, exacerbated these weaknesses, namely Fragmented, largely disciplinary structures, poorly equipped to tackle the interdisciplinary challenges posed by KTS and numbers of other modern technological problems Focus on the universities as the main performers of research and its dissemination to society, at the expense of the more balanced mix of universities and institutes used in other countries. This undermines the ability to mix short-term and strategic research activities, as was needed in KTS Levels of core funding in the applied research institutes so low that they that threaten their research mission and risk turning them into technical consultancies A focus on PhD education as the primary vehicle for doing research, leading to fragmentation, sub-critical teams and low incentives to interdisciplinarity A corresponding lack of middle-level research positions in the universities, reducing their ability to build experience, conduct large-scale research and perform their third task One of NUTEK s most important contributions has probably been to fund competence centres, which bring industrial and academic research into active oncampus partnerships and whose primary purpose is to help address some of the weaknesses in the knowledge infrastructure. Other influences, too, are encouraging the universities to modernise, so the situation is neither static nor hopeless, but more and faster reform is needed. Programmes like KTS can be part of this reform process, but they may also need to be implemented in ways that more explicitly tackle the weaknesses of the knowledge infrastructure. 8.2.3 Governance and Incentives R&D funding programmes rarely win the hearts and minds of their participants, nor is there any particularly strong reason why they should do so. In many cases, participants face a choice of funding sources and weigh up alternatives to see which have the best fit with their own needs. In the evaluation community, we increasingly talk about the real project, which is the one the participant is uses external programme money to fund. We need therefore to see projects as alliances between project performers and programme managers. These can be more or less temporary, as Exhibit 41 illustrates. 80

We can think of R&D performers real projects as defining various R&D trajectories. In some cases, these coincide with programme agendas over long periods of time. In others, the overlap may be quite brief. A key part of the programmer s problem is to design a programme that will attract enough of the right participants to overlap with the programme agenda in order to reach the programme objectives. To do this, the programme needs to offer an appropriate mix of subject focus and other incentives, which may well include various types of money. Exhibit 41 Programme Agenda and Participant Trajectories (Illustrative) Participant D Participant A Participant B Programme Participant C We saw that in KTS, while some of the participants may have been capable of making progress towards the needed generic theory, the programme lacked an adequate mechanism for encouraging them to do so in part because there was too great an industrial influence on the programme. In saying this, we are not proposing that in any future programme VINNOVA should try to find a different balance of influence between industrialists and academics, and then try to have all projects reflect this different average. Rather, there needs to be a mix of different funding instruments and degrees of influence within VINNOVA s activities. Finnish and US practice suggest possible ways to achieve this. Influence can be steered by manipulating the composition of the groups that define programmes. TEKES planning practice in electronics programmes through the 1990s provides a useful pointer to how to use stakeholders. TEKES and the Finnish electronics industry association worked together through the decade, holding a series of planning seminars with wide participation. In the early part of the 1990s when the Nokia supply chain needed rapid development, industry participants dominated the seminars and the resulting programmes focused on very short term technology acquisition and capability goals. As the industry became increasingly capable, so the proportion of researchers invited to the planing seminars role, and the agendas of the programmes became successively more ambitious. The mix of funding arrangements available in the programme is also important. A helpful funding practice is that adopted by the US military in some of its technology programmes. These have tended to be very use-oriented, but a number have also deliberately set aside 10% of their budgets for what was once described to us 57 by a 57 Erik Arnold and Ken Guy, Parallel Convergence: National Strategies in Information Technology, London: Frances Pinter, 1986 81

Pentagon official as lunatic fringe or strategic research of potential relevance to their mission. To tackle the mix of strategic and shorter term R&D needed by KTS could well, therefore, involve the use of different instruments within the same programme. (This is what KTS tried by adding its horizontal projects late in the second half of the programme, but it proved to be too little, too late.) TEKES funding practice provides one option: namely, deliberately to mix industry- and university-oriented funding within individual technology programmes. The typical TEKES technology programme has some company projects, where industry plays a strong role in agenda setting, funding and often in the performance of the project. These are often fairly close to market (as some of the KTS projects were) and the results may be secret, at least for a time. The typical programme will also have some research projects, carried out in the knowledge infrastructure and usually initiated by researchers, but with an industrial reference group monitoring 58 the activity. VINNOVA similarly needs to be able to co-ordinate industrially focused R&D with more fundamental public good research. This means either that VINNOVA should have some strategic research funds, which it can invest alongside industry co-financed projects, or that it should have close relations with other funding agencies that fund strategic or oriented basic research. The latter is not possible in today s Swedish institutional structure, with its opposition between sector agencies and researcherdirected research councils. It may be, however, that the amount of research progress needed is too big to be achieved using such relatively marginal adjustments to the funding mix. In this case a strategic research programme would make more sense. The extent of this need is, however, not a judgement we, as an evaluation team, are qualified to make. It requires expert judgement of needs and feasibility. But common to all these possible solutions is a need for VINNOVA to be involved in strong co-ordination or to have a wider range of funding instruments at its disposal, in order to be able to influence not only the development of techniques but also of technologies. Given that, in many fields, the most important scientific and technological progress is being made at the boundaries between disciplines, KTS ambition to be interdisciplinary was well justified. However, it was poorly realised partly because of the weaknesses in the knowledge infrastructure described here. The short-term industrial focus of the programme also limited its ability to tackle the real difficulties of doing interdisciplinary work. Fragmentation of the programme budget meant it was difficult to build the larger scale projects needed to achieve interdisciplinarity, and there were no clear incentives to participants to work in a way that is interdisciplinary. Policy options for VINNOVA include introducing an interdisciplinarity premium for projects where there is a good case for interdisciplinarity and where this can clearly be demonstrated. This should be accompanied by measures to build larger projects. 58 The members time is normally counted as an in-kind contribution to the project 82

8.2.4 Programme Design in an Innovation Systems Agency NUTEK and the programme board chose a deliberately technical definition of complex technical systems to be the focus of the programme. At the same time, they wanted to shape the techniques emerging from the programme so as to include important human dimensions, tackling people s behaviour as components in complex technical systems and adapting systems to human needs. They rightly identified that design the activity on which KTS focuses is a powerful place at which to exert influence over techniques. Here, there does not appear to have been a theoretical blockage. Other programmes have tackled similar issues in related technologies. But the realisation of NUTEK and the programme board s ambitions appears to have been restricted by the hardware and software focus of the engineering knowledge used, the lack of end user influence in the programme and, again, the comparatively short-term industrial focus of the projects. The technical focus of the programme no doubt made sense within NUTEK Teknik, but in the new context of VINNOVA as the national agency for innovation systems, it would be more positive to extend the definition to take on the roles of complex technical systems within the innovation system. This suggests, on the more technical side, greater concern with techniques for linking complexity to the very different user requirements in complex product systems and in more mass-produced complex products. It also suggests that projects could be clustered around the very different supply chains active in these two areas. We do not suffer from the illusion that there will ever be a generic and permanently valid description of how an optimal innovation system works. Innovation as a process works by constantly changing the rules of the game. The successful innovator redefines reality, and the next innovation therefore takes place under new rules. A generic description of how the innovation system works will not, therefore, stay valid. Rather, it will serve as a challenge to creativity in changing it. Correspondingly, VINNOVA will need constantly to monitor and analyse the situation in individual fields and areas to identify weaknesses and improvement opportunities. 59 Certainly, it will be helpful to define programme needs through a combination of technical and innovation systems analysis: moving, in this case, from a complex technical systems perspective to one which considers the role of technology in the innovation system, as in the distinction made earlier between CoPS and mass-produced, stand-alone complex products. This has important implications for the programme design process. It is not enough simply for NUTEK/VINNOVA to act as a broker among the various interested stakeholders. This is likely to cause the kind of lock-ins to current problems that kept the focus of KTS too industrial. Rather, it will make sense to couple analytic capabilities to individual programmes or areas of effort, in order to secure a balance between analytic and stakeholder input. Good analysis here will include understanding of both the technical and the economic issues involved. However, programme definitions that are wholly driven by analysis and which ignore 59 Elsewhere, we describe this as bottleneck analysis in the National Innovation System. See Erik Arnold, A systems world needs systems evaluations, Research Evaluation (forthcoming 2003) 83

stakeholders will fail just as surely as stakeholder-driven programmes that ignore the facts. 8.3 Next Steps VINNOVA is grappling with the question: What, if anything, should follow on from the KTS programme? A workshop on the subject, held by NUTEK in late 2000, produced a large number of short position papers. There was a wide diversity of ideas, with recurring themes being Generic complexity Human factors and the human dimension Complex software systems development Subsequent internal VINNOVA proposals for follow-on programmes include SAMBA (also referred to as Functional and user-friendly complex technical systems), which aims to make the complex systems world comprehensible and manageable for human beings, in their various role, and give them full control of systems. This idea steps directly up to the challenges left behind by KTS, in terms of low goal attainment in generic systems knowledge and integration people and human factors into work on complex systems Industrial measurement systems, aiming to integrate IT, biotechnology, materials science and nanotechnology in the context of improved industrial measurement and control The questionnaire used in this evaluation asked participants for their views on issues relating to the need for future support in areas covered by the previous programme. For each technology area, respondents were asked to rate the strength of Swedish research capability, the strength of demand for new tools and methods, the ability of industry to exploit research results and (hence) the strength of case for new or continued funding. The results obtained are shown in Exhibit 42 below. Except for the area of optimising complex infrastructures, where needs might be high but capabilities to research and deliver were rather lower, the scores from the survey do not discriminate a great deal among the themes. Complex systems development and systems architecture are the most needed, but the research system is best able to deliver complex modelling and simulation. This is consistent with what we know of the knowledge infrastructure and the fact that the first two themes are more fundamentally industrial than modelling and simulation. If the respondents judgement about the need for better tools in relation to complex infrastructure optimisation is correct, then there may be a case for a more strategic intervention to build up research capabilities. 84

Exhibit 42 Future Policy Mean ratings on 1-5 scale (n=60) Strength of Swedish research capability Strength of demand for new methods / tools Ability of Swedish industry to exploit results Strength of case for new/continued funding Complex systems development 3.78 4.35 3.55 4.05 Complex systems architecture 3.86 4.29 3.63 4.04 Complex decision support systems 3.46 3.94 3.19 3.53 Complex modelling and simulation 4.09 4.09 3.53 3.97 Optimisation of complex infrastructures 3.31 4.13 3.24 3.28 Respondents were also asked to identify other areas where there was a strong case to be made for public funding. The list below shows the suggestions that have been received to date. We read this list largely as an invitation to VINNOVA to carry on funding whatever the individual questionnaire respondent happens to do or to want to do next. Exhibit 43 Questionnaire Respondents Proposed Future Programme Themes Complex Control of Electro-Mechanical Systems Complex Embedded Systems Complex Management of Data Complex Mechanical Systems Complex Process Control Complex Process Optimization Distributed complex system development Distributed Control of Complex Systems Dynamic distributed Database Tech Empirical Software Engineering evolutionary algorithms Evolutionary robotics Flexibility in complex system development Humanoid robotics Informal and semiformal semantic modelling and Information fusion design IT Integrity in Networks Management of complex system development Management of development in networks of organisations Methodology and technology for co-operation in network Modeling of dynamics in Energy-, Power and Network Based System Co-ordination Process Systems (Integrated dynamic system simulations for development and training) Process-oriented systems integration Robotics Sensor Data Integration in Autonomous Robotics Software Evolution and Maintenance Software Process Research Systems engineering and computational design Systems of complex technical systems Task-level Programming and Sensor Interfaces Verification and testing of complex systems Source: Questionnaire survey conducted for this evaluation It is clear that the KTS programme leaves behind unresolved questions whose socioeconomic importance is large. Our evaluation implies that running a further programme in the same way as before is unlikely to resolve these questions. Much of the discussion about continuation is focused on research and/or technology. Modern theory says that innovations occur when socio-economic needs and technological opportunities are coupled together. Perhaps VINNOVA should take its own innovation systems medicine and try to use this principle in developing a 85

potential new programme design to address what clearly remains an area of huge social and economic importance in Sweden. Papers at the 2000 KTS workshop from Karl-Erik Årzén, LTU, and Jakob Axelsson, Volvo Teknisk Utveckling were unusual in that they went in this direction. Axelsson s was especially interesting, because it explicitly discussed longer-term trends in business and engineering, how they affected Volvo and the types of generic research question that needed tackling in order to increase Volvo s competitiveness. The issues Axelsson raised 60 were neither trivial not particularly specific to Volvo. They were Decision making. How do we improve our ability systematically to take technical (and other) decisions at early stages of the design process and in a synthetic way, when the problems to be tackled are vague, multidisciplinary, complex and with built-in contradictions, and there are many potential but uncertain solutions? Information processing. How should we handle the huge quantity of information that is related to a complex vehicle system as efficiently as possible, during and after the development project? Global cooperation. How can we turn the difficulties of doing development work globally into a source of strength that increases quality and reduces development times? Productivity. How can we reduce development times at the same time as complexity is increasing? Modelling. How can we improve our ability to use computer models to predict the characteristics of an intended heterogeneous system and synthesise its design? Integration. How can we assemble advanced mechanical engineering, computer hardware, software etc from a large number of suppliers into a functioning system that can be produced and maintained, and which works efficiently with people? What platforms and architectures are needed as bases for work on these problems? These types of questions are especially relevant to the complex, stand-alone products, quadrant of Exhibit 40. Other types of question will be more relevant for complex product systems. It would be possible to focus a new programme design by soliciting these kinds of longer-term business needs driven questions and confronting them with a set of more research-driven ones. The interesting focus for a new programme would then be in the overlaps and in the areas adjacent to them. This initial analysis could probably best be done in a moderated workshop involving people from the research community and from some of the (mostly) large companies that have the business and technology capabilities to engage seriously in this level of discussion. Even though there has been a strong desire from the political side since the early 1990s to focus innovation policy increasingly on SMEs, there are areas where it is mostly large companies that have the capabilities to define problems and implement solutions. Because of the scale of the phenomena, complex technical systems are one such area. 60 Jakob Axelsson, Standardisering eller diversifiering? Utmaningar inom utvecklingen av komplexa fordonssystem, Volvo Teknisk Utveckling, 11 October 2000 86

The additional element required is analysis. Before trying to launch a programme, it is necessary to Understand the state of the art, to make sure that the answers to the programme s research questions are not already available or that the proposals do not introduce some new mission impossible to the agenda Scope the potential areas of social and economic impact. This is not to say that an economic impact or cost-benefit analysis should be attempted, for this is methodologically untenable. Rather, it is important simply to confirm that the intended action is likely to affect large and important parts of the Swedish innovation system rather than small or unimportant ones It would make sense to complement these analyses with an explicit programme activity that maintains a technology watch on foreign developments. Sweden may be an R&D-intensive country, but she still performs only about 1% of the world s R&D, and the people doing the other 99% are not stupid. In the short term, VINNOVA would have at least three options in terms of the type of programme to put in place 1 A combined co-funded industrial programme and a 100%-funded more researchintensive strand, somewhat in the style of TEKES 2 A call for tenders to establish two or three new competence centres or, perhaps, a single, much larger multi-site competence centre. The Austrian K+ programme could be a useful model here, since it has modified the Swedish competence centre model to operate at larger scale up to 50 or so researchers, some from industry and some from the knowledge infrastructure. It also has a structure that tends top protect a long-term research component within each centre 3 An invitation to construct an industry consortium or consortia, working with long term research questions in the area of complex technical systems. The MediaLab at MIT could provide a model, where companies buy into research programmes for extended periods. IMEC, the major microelectronics research institute in Flanders, works in a similar way The main criteria for choosing among these forms is the extent to which the problems identified are likely to persist, the extent to which tackling them involves cutting across existing departmental lines in the universities and the willingness of industry to become involved as a long-term partner. In the longer term, VINNOVA has an opportunity to use the complexity agenda as a platform for negotiating a pattern of working together with research councils and foundations that goes beyond today s rather minimal exchange of information. This would be the start of a rather ambitious process of co-ordination in an overfragmented R&D funding system. But then there is no end without a beginning. 87

Appendix A Questionnaires to Project Participants 88

Questionnaire for Research-base Participants Evaluation of the Complex Technical Systems (CTS) Programme This questionnaire is part of a study conducted for Vinnova by Technopolis Ltd., UK. Your cooperation in answering the questions is kindly requested. All individual answers and comments will be treated as strictly confidential and non-attributable. Please use a separate questionnaire for each project in which you are involved. In addition, unless otherwise specified, please answer for your organisation's participation in each project, and not for the project as a whole. NAME OF RESPONDENT NAME OF ORGANISATION TEL/FAX EMAIL PROGRAMME AREA A1 Systems Development A4 Decision Support Systems B1.1 IT Integrity B4.1 Sensor Data Fusion B1.2 Software Engineering A5 Modelling and simulation B1.3 Integrated Product Data Mgmt A6 Optimisation of Infrastructures A2 System Architecture Horizontal PROJECT TITLE TIMING OF PROJECT Start Date Duration (months) NUTEK FUNDING For your organisation's participation SEK For the project as a whole SEK TOTAL FUNDING For your organisation's participation SEK For the project as a whole SEK Please email the completed questionnaire to Technopolis at james.stroyan@technopolis-group.com. Alternatively you can fax it on +44 1273 747299 or mail it to Technopolis Ltd, 3 Pavilion Buildings, UK - Brighton BN1 1EE. For further information please contact James Stroyan at the above address, or by telephone on +44 1273 204320. Alternatively, speak to Anders Hedin at Vinnova (Tel: + 46 8 473 3126; Fax: + 46 8 473 3005; E-mail: anders.hedin@vinnova.se).

A. ABOUT YOUR PROJECT 1. Please indicate whether or not your project follows on from or exploits work in earlier STU or NUTEK-funded projects. Yes No 2. Please indicate the strategic importance of this project to your organisation Minor Major 3. Please tick each of the following scales to characterise the nature of your organisation s participation in the project. Low cost High cost Low risk High risk Technically simple Technically complex Mundane Exciting Necessary A luxury Short-term Long-term Fundamental Applied Curiosity-driven Mission-oriented Oriented to a specific complex problem Oriented to generic issues of complexity R&D-oriented Production-oriented Product-oriented Process-oriented Software-oriented Hardware-oriented Idea came from research community Idea came from industrial community In a core technology area for your organisation In a peripheral technology area for your organisation Strongly connected to other in-house projects Not connected to other in-house projects 4. What would have been the impact on this project if it had not received NUTEK funding? Project would have gone ahead unchanged Your organisation would not have been able to undertake the project at all Your organisation would have undertaken the project, but: With reduced objectives With reduced funds With longer time scale With fewer partners With international collaboration Other (please specify )

B. MOTIVES, GOALS, OUTPUTS and OUTCOMES 5. For your organisation's participation in this project: How important were the following as motives and goals? To what extent were expected outputs and outcomes achieved? MOTIVES and GOALS Not important Very important OUTPUTS and OUTCOMES Less than expected More than expected Knowledge-oriented goals Enhancement of knowledge base in core technology areas Enhancement of knowledge base in new, alternative technology areas Increased understanding of new methods and tools Increased skills of research staff Acceleration of R&D Reorientation of R&D portfolio towards longer-term R&D Reorientation of R&D portfolio towards shorter-term R&D 1 2 3 4 5 1 2 3 4 5 Networking-oriented goals Access to complementary sources of expertise Formation of new research partnerships and networks Better co-operation with businesses Follow-on entry into other national R&D programmes Follow-on entry into international R&D programmes Follow-on entry into R&D collaborations in the private sector Follow-on entry into R&D collaborations with universities / research institutes Exploitation-oriented goals Development or improvement of new processes Development or improvement of new products or services Development, evaluation or improvement of tools and techniques Improved competitiveness vis-à-vis domestic competitors Improved competitiveness vis-à-vis foreign competitors Increased turnover, market share or productivity Production of patents and licences Strategic management-oriented goals Access to additional funds Cost-sharing between partners Monitoring of competitors' activity Enhanced reputation and image Reduced risk of R&D

C. FACTORS AFFECTING PROGRESS AND EXPLOITATION 6. Please indicate the type of impact the following factors had on your participation in this project and on the exploitation of project results by your organisation. IMPACT ON PROJECT PROGRESS IMPACT ON EXPLOITATION Very Negative Negative No Impact Positive Very Positive Very Negative Negative No Impact Positive Very Positive Competence of own organisation Competence of other participants Competence of NUTEK programme officials Activities of programme steering committee Adequacy of goal specification Process of drawing up consortium agreements The nature of the consortium agreements Structure of the project Structure of the programme Complexity of technical issues addressed Interdisciplinary issues/complexity within project team Restructuring / strategic changes in own organisation Restructuring / strategic changes in other project partners Changing global technological trends Availability of qualified personnel Access to sufficient amounts of project funding Level of interest within own organisation Level of interest within other participants Level of interest within potential users Delays in technology development within the project Delays in technology development outside the project Adequacy of project outputs D. NUTEK PROCEDURES 7. How would you describe the input you received from the Programme Director, the NUTEK Programme Contact or the Programme Steering Committee concerning your project? Prior to submitting your proposal Unhelpful/ negative Neutral Helpful Very helpful No input During project selection and negotiation During the life of your project Feedback and follow-up to your project

8. How do you assess NUTEK procedures for making an application for research funding? Easy to follow Clear documentation/ information leaflets Quick Difficult to follow Inadequate or too complex documentation/information leaflets Slow E. OUTPUTS AND RESULTS 9. How important is each of the following outputs in assessing the success of the project? Essential Important Secondary Unimportant Publications in refereed journals Other publications PhD theses Patent applications Patents granted New products New processes New methods or tools Pilots or prototypes Norms and standards Other (please specify below) 10. To what extent has your project made a contribution to the following? Improved systems development capabilities within the university / research institute sector Improved systems development capabilities within industry Development of tools and methods for complex technical system development Dissemination of information on new methods and tools via demonstrations / applications Transfer of capabilities between desciplines and sectors Transfer of technologies between disciplines and sectors No contribution Major contribution 1 2 3 4 5 F. COSTS AND BENEFITS 11. Overall, how do the costs and benefits associated with your organisation's participation in this project balance out? Please tick one of the following boxes. Costs outweigh benefits Costs equal benefits Benefits outweigh costs -3-2 -1 0 1 2 3

G. SELF ASSESSMENT 12. Please consider your project in terms of the following and provide an overall rating for each. The adequacy of the project agenda / workplan The adequacy of the project resources in terms of staff, equipment, finances, infrastructure, etc. The organisation and management of the project The quality of the project outputs The extent to which the project has made significant contributions to knowledge The extent to which the projects technical and exploitation goals were attained The utility of the project results to your organisation The extent to which the project has been exploited within your organisation The utility of the project results to other businesses / research institutions The extent to which the project has been exploited within other business / research institutions Very low/ poor Very high/ good 1 2 3 4 5 H. FUTURE 13. The areas below correspond to the current CTS programme areas. For each area, please use 1-5 scales (1=low; 5=high) to indicate The strength of Swedish research capability The strength of demand for new methods and tools The ability of Swedish industry to exploit developments The strength of the case for new / continued funding Please answer for as many areas as possible, not just those in which you are involved. Strength of Swedish research capability Strength of demand for new methods / tools Ability of Swedish industry to exploit results Strength of case for new/continued funding Complex Systems Development Complex Systems Architecture Complex Decision Support Systems Complex Modelling and Simulation Complex Optimisation of Infrastructures 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 If there are any other related areas where there is a strong case for the launch of new sub-programmes, please list them below

14. What happened/will happen at the end of your project? The project team will... Abandon research efforts in the area Continue the research in another Vinnova programme Continue the research with own resources Pursue the research with other national funding (e.g. research council) Pursue the research with international funding (e.g. EC) Begin new collaborative ventures together Begin commercialisation of project results Other (please specify) 15. Please provide any additional comments you may have on any aspect of the CTS programme. THANK YOU FOR TAKING THE TIME AND TROUBLE TO COMPLETE THIS QUESTIONNAIRE

Questionnaire for Industrial Participants Evaluation of the Complex Technical Systems (CTS) Programme This questionnaire is part of a study conducted for Vinnova by Technopolis Ltd., UK. Your cooperation in answering the questions is kindly requested. All individual answers and comments will be treated as strictly confidential and non-attributable. Please use a separate questionnaire for each project in which you are involved. In addition, unless otherwise specified, please answer for your company's participation in each project, and not for the project as a whole. NAME OF RESPONDENT COMPANY NAME TEL/FAX EMAIL PROGRAMME AREA A1 Systems Development A4 Decision Support Systems B1.1 IT Integrity B4.1 Sensor Data Fusion B1.2 Software Engineering A5 Modelling and simulation B1.3 Integrated Product Data Mgmt A6 Optimisation of Infrastructures A2 System Architecture Horizontal PROJECT TITLE TIMING OF PROJECT Start Date Duration (months) NUTEK FUNDING For your company s participation SEK For the project as a whole SEK TOTAL FUNDING For your company s participation SEK For the project as a whole SEK SIZE OF COMPANY Firm with < 50 employees Firm with 250-499 employees Firm with 50-99 employees Firm with 100-249 employees Firm with > 500 employees Other (please specify below) Please email the completed questionnaire to Technopolis at james.stroyan@technopolis-group.com. Alternatively you can fax it on +44 1273 747299 or mail it to Technopolis Ltd, 3 Pavilion Buildings, UK - Brighton BN1 1EE. For further information please contact James Stroyan at the above address, or by telephone on +44 1273 204320. Alternatively, speak to Anders Hedin at Vinnova (Tel: + 46 8 473 3126; Fax: + 46 8 473 3005; E-mail: anders.hedin@vinnova.se).

A. ABOUT YOUR PROJECT 1. Please indicate whether or not your project follows on from or exploits work in earlier STU or NUTEK-funded projects. Yes No 2. Please indicate the strategic importance of this project to your company Minor Major 3. Please tick each of the following scales to characterise the nature of your company s participation in the project. Low cost High cost Low risk High risk Technically simple Technically complex Mundane Exciting Necessary A luxury Short-term Long-term Fundamental Applied Curiosity-driven Mission-oriented Oriented to a specific complex problem Oriented to generic issues of complexity R&D-oriented Production-oriented Product-oriented Process-oriented Software-oriented Hardware-oriented Idea came from senior management Idea came from research staff In a core technology area for your company In a peripheral technology area for your company Strongly connected to other in-house projects Not connected to other in-house projects 4. What would have been the impact on this project if the project had not received NUTEK funding? Project would have gone ahead unchanged Your company would not have been able to undertake the project at all Your company would have undertaken the project, but: With reduced objectives With reduced funds With longer time scale With fewer partners With international collaboration Other (please specify )

B. MOTIVES, GOALS, OUTPUTS and OUTCOMES 5. For your company's participation in this project: How important were the following as motives and goals? To what extent were expected outputs and outcomes achieved? MOTIVES and GOALS Not important Very important OUTPUTS and OUTCOMES Less than expected More than expected Knowledge-oriented goals Enhancement of knowledge base in core technology areas Enhancement of knowledge base in new, alternative technology areas Increased understanding of new methods and tools Increased skills of research staff Acceleration of R&D Reorientation of R&D portfolio towards longer-term R&D Reorientation of R&D portfolio towards shorter-term R&D 1 2 3 4 5 1 2 3 4 5 Networking-oriented goals Access to complementary sources of expertise Formation of new research partnerships and networks Better co-operation with universities and research institutes Follow-on entry into other national R&D programmes Follow-on entry into international R&D programmes Follow-on R&D collaborations in the private sector Follow-on R&D collaborations with universities / research institutes Exploitation-oriented goals Development or improvement of new processes Development or improvement of new products or services Development, evaluation or improvement of tools and techniques Improved competitiveness vis-à-vis domestic competitors Improved competitiveness vis-à-vis foreign competitors Increased turnover, market share or productivity Production of patents and licences Strategic management-oriented goals Access to additional funds Cost-sharing between partners Monitoring of competitors' activity Enhanced reputation and image Reduced risk of R&D

C. FACTORS AFFECTING PROGRESS AND EXPLOITATION 6. Please indicate the type of impact the following factors had on your participation in this project and on the exploitation of project results by your company. IMPACT ON PROJECT PROGRESS IMPACT ON EXPLOITATION Very Negative Negative No Impact Positive Very Positive Very Negative Negative No Impact Positive Very Positive Competence of own company Competence of other participants Competence of NUTEK programme officials Activities of programme steering committee Adequacy of goal specification Process of drawing up consortium agreements The nature of the consortium agreements Structure of the project Structure of the programme Complexity of technical issues addressed Interdisciplinary issues/complexity within project team Restructuring / strategic changes in own company Restructuring / strategic changes in other project partners Changing global technological trends Availability of qualified personnel Access to sufficient amounts of project funding Level of interest within own company Level of interest within other participants Level of interest within potential users Delays in technology development within the project Delays in technology development outside the project Adequacy of project outputs D. NUTEK PROCEDURES 7. How would you describe the input you received from the Programme Director, the NUTEK Programme Contact or the Programme Steering Committee concerning your project? Prior to submitting your proposal Unhelpful/ negative Neutral Helpful Very helpful No input During project selection and negotiation During the life of your project Feedback and follow-up to your project

8. How do you assess NUTEK procedures for making an application for research funding? Easy to follow Clear documentation/ information leaflets Quick Difficult to follow Inadequate or too complex documentation/information leaflets Slow E. OUTPUTS AND RESULTS 9. Has the project already led to economic benefits for your company? Yes No If yes, please describe the nature of the new or improved product / service / system and explain how the project results have contributed to recent developments. Please also indicate the scale of the economic benefits to date. 10. Does your company have any plans for the future commercial exploitation of project results? Yes No If yes, please describe the nature of the planned product / service / system, the expected time to market and any critical success factors in terms of successful exploitation 11. To what extent has your project made a contribution to the following? Improved systems development capabilities within the university / research institute sector Improved systems development capabilities within industry Development of tools and methods for complex technical system development Dissemination of information on new methods and tools via demonstrations / applications Transfer of capabilities between desciplines and sectors Transfer of technologies between disciplines and sectors No contribution Major contribution 1 2 3 4 5 F. SELF ASSESSMENT 12. Please consider your project in terms of the following and provide an overall rating for each. The adequacy of the project agenda / workplan The adequacy of the project resources in terms of staff, equipment, finances, infrastructure, etc. The organisation and management of the project The quality of the project outputs The extent to which the project has made significant contributions to knowledge The extent to which the projects technical and exploitation goals were attained The utility of the project results to your company The extent to which the project has been exploited within your company The utility of the project results to other businesses / research institutions The extent to which the project has been exploited within other business / research institutions Very low/ poor Very high/ good 1 2 3 4 5

G. COSTS AND BENEFITS 13. Overall, how do the costs and benefits associated with your company's participation in this project balance out? Please tick one of the following boxes. Costs outweigh benefits Costs equal benefits Benefits outweigh costs -3-2 -1 0 1 2 3 H. FUTURE 14. What happened/will happen at the end of your project? The project team will... Abandon research efforts in the area Continue the research in another Vinnova programme Continue the research with own resources Pursue the research with other national funding (e.g. research council) Pursue the research with international funding (e.g. EC) Begin new collaborative ventures together Begin commercialisation of project results Other (please specify) 15. The areas below correspond to the current CTS programme areas. For each area, please use 1-5 scales (1=low; 5=high) to indicate The strength of Swedish research capability The strength of demand for new methods and tools The ability of Swedish industry to exploit developments The strength of the case for new / continued funding Please answer for as many areas as possible, not just those in which you are involved. Strength of Swedish research capability Strength of demand for new methods / tools Ability of Swedish industry to exploit results Strength of case for new/continued funding Complex Systems Development Complex Systems Architecture Complex Decision Support Systems Complex Modelling and Simulation Complex Optimisation of Infrastructures 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 If there are any other related areas where there is a strong case for the launch of new sub-programmes, please list them below

16. Please provide any additional comments you may have on any aspect of the CTS programme. THANK YOU FOR TAKING THE TIME AND TROUBLE TO COMPLETE THIS QUESTIONNAIRE

Appendix B Questionnaire Non-Response Analysis We explored the responding and non-responding populations to see whether there were important differences. Two potential sources of bias emerge. One is that nonrespondents were more likely to come from the Systems Development subprogramme. The other is that people from single-organisation projects were least likely to respond. Exhibit 44 area? Are respondents more likely to belong to a particular programme Respondents/non-respondents by programme area 25 20 Respondents Non-respondents 15 10 5 0 A1 A2 A3 A4 A5 A6 H1 H2 Programme Area There appears to be no significant relationship between programme area and likelihood of response, however many of the populations are quite small. None of the four participants in A6 (Optimisation of Infrastructures) responded to the survey. Only in A4 (Decision Support Systems) did more than half the participants respond. The larger sub-programmes A1 and A5 had less than 50% response rate. 103

Exhibit 45 Number of Partners in Responding Projects Response/non-response vs no. of Partners 16 14 12 10 8 6 Respondents Non-respondents 4 2 0 1 2 3 4 5 6 7 8 9 10 No of partners It appears that single-organisation projects were much less likely to provide a response. Of those cohorts with a significant population (i.e. more than ten projects) response was most likely to be received from projects with 2 or 3 partners. 104

B.1 Additional Questionnaire Analyses Nature of Projects Low cost - High cost Low risk - High risk 50% 40% 30% 20% 10% 0% 1 2 3 4 5 40% 30% 20% 10% 0% 1 2 3 4 5 Simple - Complex Mundane - Exciting 50% 40% 30% 20% 10% 0% 1 2 3 4 5 50% 40% 30% 20% 10% 0% 1 2 3 4 5 Necessary - Luxury Short-term - Long-term 40% 60% 30% 20% 10% 0% 1 2 3 4 5 40% 20% 0% 1 2 3 4 5 Fundamental - Applied Curiosity - Mission-oriented 60% 80% 40% 20% 60% 40% 20% 0% 1 2 3 4 5 0% 1 2 3 4 5 105

Specific problem - Generic problem R&D - Production 40% 30% 20% 10% 0% 1 2 3 4 5 50% 40% 30% 20% 10% 0% 1 2 3 4 5 Product - Process Software - Hardware 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 50% 40% 30% 20% 10% 0% 1 2 3 4 5 Senior mgmt idea - Researcher idea Core tech area - Peripheral tech area 30% 25% 20% 15% 10% 5% 0% 1 2 3 4 5 50% 40% 30% 20% 10% 0% 1 2 3 4 5 Strong connect to in-house - Not connected to in-house 50% 40% 30% 20% 10% 0% 1 2 3 4 5 106

Exhibit 46 Respondents Goal Attainment by Ranked Motives and Goals Area Motive Rating Achieved less Achieved what Achieved more than expected expected than expected Knowledge Increased understanding of new methods and tools 4.02 8% 29% 63% Knowledge Enhancement of existing knowledge base 3.61 6% 52% 42% Networking Formation of new research partnerships and networks 3.53 11% 30% 60% Networking Access to complementary sources of expertise 3.41 11% 34% 55% Knowledge Increased skills of research staff 3.40 7% 46% 48% Networking Better co-operation with universities and research institutes 3.37 23% 29% 48% Exploitation Development, evaluation or improvement of tools and techniques 3.37 23% 46% 31% Exploitation Development or improvement of new processes 3.29 15% 39% 46% Knowledge Acceleration of R&D 3.22 18% 47% 36% Knowledge Enhancement of knowledge base in new, alternative technology areas 2.98 22% 39% 39% Strategic management Enhanced reputation and image 2.92 14% 45% 41% Networking Follow-on R&D collaborations in the private sector 2.82 33% 40% 27% Networking Follow-on R&D collaborations with universities / research institutes 2.75 22% 52% 26% Exploitation Improved competitiveness vis-à-vis foreign competitors 2.69 21% 57% 21% Strategic management Access to additional funds 2.65 30% 39% 30% Exploitation Development or improvement of new products or services 2.65 26% 51% 23% Strategic management Cost-sharing between partners 2.53 25% 45% 30% Networking Follow-on entry into other national R&D programmes 2.53 30% 46% 24% Exploitation Increased turnover, market share or productivity 2.53 38% 47% 15% Networking Follow-on entry into international R&D programmes 2.33 28% 48% 24% Knowledge Reorientation of R&D portfolio towards longer-term R&D 2.31 19% 60% 21% Strategic management Reduced risk of R&D 2.29 26% 49% 26% Exploitation Improved competitiveness vis-à-vis domestic competitors 2.25 22% 61% 17% Knowledge Reorientation of R&D portfolio towards shorter-term R&D 1.67 24% 71% 5% Strategic management Monitoring of competitors' activity 1.63 35% 57% 9% Exploitation Production of patents and licenses 1.39 35% 50% 15%

Appendix 1 (in Swedish)/Bilaga 1 Om Komplexa tekniska system - programmens bakgrund, syften och genomförande 1. Bakgrund och behovsbild Programverksamheten inom Komplexa tekniska system har byggts upp successivt från en bred behovsbild inom både industri och universitet. I slutet av 80-talet genomförde svensk industri och staten i samverkan massiva insatser i form av Nationella mikroelektronikprogrammet följt av det systemtekniskt inriktade IT4- programmet. 1990/91 var dessa i huvudsak avslutade men oron var stor att svenska industriella behov av metod- och systemkompetens inte skulle kunna tillgodoses i den allt snabbare globala systemutvecklingen. 1991 tillsatte regeringen en snabbt arbetande utredning, IT2000, som redan i oktober samma år levererade ett antal utredningsrapporter och sina slutsatser. Systemteknik fördes här fram som ett viktigt område med en fokusering på stora system. Det ansågs att kompetens för att hantera stora system är en bristvara i alla de stora företagen samtidigt som just systemkompetensen är ett av de svenska företagens viktigaste konkurrensmedel. Parallellt pekades systemteknik för komplexa system ut som ett viktigt område för svensk forskning i svaren på NUTEKs breda enkät till forskarsamhället 1991. Denna enkät följdes under våren 1992 av ett större IVA/NUTEK-seminarium med temat Komplexa system en framtida allmän teknologi. Det mycket kompetenta auditoriet vid seminariet diskuterade många frågeställningar som kom att ha stor betydelse i de kommande programmens verksamhet. Behovet av insatser inom området ansågs stort, dels beroende på de allt starkare behoven av interdisciplinär kompetens och dels beroende på behovet av inbyggd flexibilitet som kunde underlätta framtida förändring och utveckling av systemen. Också behovet av helhetsgrepp ansågs stort. Många diskuterade vid seminariet balansen mellan långsiktig forskning och alltför kortsiktigt industriellt utvecklingsarbete. Vissa ansåg att området inte var forskningsbart och inte heller undervisningsbart, men flertalet menade att man underifrån kunde bygga metodutvecklingen från en stark koppling till applikationen upp till en generellare nivå. En intressant fråga, som programmet egentligen har brottats med under hela verksamheten, är definitionen av ett komplext system. Den mest salomoniska lösningen vid seminariet var Du känner igen det när Du ser det. I övrigt var man eniga om att det var en väsentlig skillnad mellan komplicerade och komplexa system. 1

Appendix 1 (in Swedish)/Bilaga 1 2. NUTEKs första program inom Komplexa system 1993-1997 Programplanering De nämnda utredningarna och seminariet utgjorde sommaren 1992 tillsammans med interna NUTEK-arbeten en grund för NUTEKs anslagsframställan för budgetåren 1993 96. NUTEK föreslog att en specifik programinsats rörande komplexa system skulle byggas upp successivt under programperioden 1993-96 för att beroende på bl.a. industrins intresse och engagemang kunna förstärkas och breddas under perioden. NUTEKs avsikt med ett program rörande komplexa system beskrevs vara att bearbeta problem som var av vital betydelse för svenskt näringsliv. Resultaten skulle vara till nytta för flera branscher, verksamheter eller företag. Detta ansågs kunna åstadkommas genom att programmet gavs en tyngd mot allmängiltiga metoder. Programmet krävde en bred förankring med en medverkan från flera kompetensområden och förutsatte en aktiv samverkan mellan näringsliv och forskningsinstitutioner eventuellt i form av gemensamma konsortier av t.ex. den typ som tillämpades inom EG:s forskningsprogram. Efter att forskningspropositionen 1992/93:170 hade presenterats i början av 1993 kunde NUTEK arbeta vidare med en brett förankrad programutveckling. En intern NUTEK-grupp utarbetade en preliminär programplan som presenterades och ingående diskuterades vid ett seminarium med cirka 100 deltagare från industriföretag, institut och högskolor den 24 februari 1993. De framförda synpunkterna utgjorde underlag för framtagning av en slutlig plan för programmet Systemteknik och utvecklingsmetodik för komplexa tekniska system. I arbetet med denna plan deltog en extern planeringsgrupp bestående av personer från Ericsson, Saab Scania, Volvo, FMV, STFI och SMHI. För att det skulle bli lättare att hitta synergieffekter och möjligheter till samverkan strukturerades programmet i tre problemområden: Utvecklings- och konstruktionsmetodik för tekniska systemprodukter Teknik för konstruktion, drift och underhåll av komplexa produktions- och processystem Strukturering av distributionssystem och nätverk Målet för programmet var att finna generella metoder, verktyg, beskrivningsmodeller mm för att utveckla, hantera, producera och använda (i positiv betydelse) komplexa system och därvid bemästra den därmed (i negativ betydelse) förknippade komplexiteten. Programmet utformades så att det skulle kunna genomföras i samarbete med högskola, institut och näringsliv. Genomförande Eftersom programmet förutsatte en aktiv samverkan mellan näringsliv och forskningsinstitutioner i konsortieform och det krävs tid och arbete för att utveckla nya nätverk, beslutades att projektförslagen skulle utformas i två steg. I första omgången (våren -93) skedde utlysning och beslutades om planeringsbidrag för detaljutformning av projektförslag och etablering av projektgrupper. I nästa omgång (hösten -93) inlämnades och 2

Appendix 1 (in Swedish)/Bilaga 1 bedömdes de fullständiga projektförslagen. Intressenterna informerades om att konsortierna borde redovisa planer för 3-6 år men att man i första omgången bara skulle få beslut för första året. De slutliga projekt- och konsortieförslagen inlämnades i oktober 1993 och behandlades av en styrgrupp bestående av företrädare för några stora systemföretag och forskare vid högskoleinstitutioner med Lars Göran Rosengren, Volvo, som ordförande. Bedömningen av förslagen baserades på ett antal kriterier som styrgruppen diskuterat fram. Vid sidan av de mera konventionella bedömningskriterierna rörande industriell relevans och ekonomisk potential samt vetenskaplig kvalité och kompetens hade styrgruppen enats om ett antal programspecifika kriterier, som betonade multidisciplinär samverkan i projekten och mellan högskola, institut och näringsliv. Det betonades också att de framtagna metoderna skulle vara vetenskapligt baserade och kunna användas inom flera tillämpningsområden. Styrgruppens behandling av projektförslagen, som innefattade en muntlig föredragning av konsortiet, ledde fram till att beslut togs om att starta 13 projekt från 1 januari 1994. Eftersom industriföretagen skulle matcha NUTEKs bidrag, föredrog styrgruppen bl.a. av ansvars- och redovisningsskäl att industriföretagen var anslagsmottagare. För att skapa erfarenheter av olika arbetssätt tilläts det dock i några fall att högskolan var anslagsmottagare och även projektledare, men då med ansvar att redovisa en industriell matchning som kunde revideras. Styrgruppen fann vid sin projektbedömning att flertalet projekt inte tog upp human factors i den utsträckning som styrgruppen hade önskat. Man reserverade därför medel som efter ansökan kunde få disponeras för riktade insatser mot human factors. Tyvärr var styrgruppen också tvungen att konstatera att den traditionella processindustrin lyste med sin frånvaro bland de projektförslag som hållit måttet inför beslutskriterierna. Under 1994 gjordes därför speciella ansträngningar för att få med sådana förslag i programmet. Tyvärr visade sig processindustrin vara tämligen ointresserad av de bredare angreppssätt över branscher och discipliner som programmet förutsatte. Utifrån de syften och mål som angivits för programmet hade styrgruppen på ett tidigt stadium enats om att det arbetssätt som skulle användas, var att basera metodutvecklingen på arbete med verkliga tillämpningsexempel för att sedan generalisera metoderna och använda dem för andra tillämpningar. Nätverksbyggandet och samverkan tvärs discipliner och branscher fick hög prioritet och under arbetets gång ordnades årliga programkonferenser där denna samverkan stimulerades genom lämpligt valda teman och arbetskonstellationer. Utvärdering Redan från programmets start fanns avsikten att det, om resultatet var positivt, skulle kunna fortsätta ytterligare någon treårsperiod och eventuellt förstärkas. Våren 1996 genomfördes en utvärdering av programmets industrirelevans av en dansk och en finsk utvärderare (NUTEK-rapport R 1996:61) Utifrån programmets egna syften och mål men 3

Appendix 1 (in Swedish)/Bilaga 1 också utifrån studier av svensk och europeisk industripolitik bedömde utvärderarna programmet som framgångsrikt med följande övergripande slutsats: The Programme System Techniques and Development Methodologies for Complex Technical Systems is addressing a set of important generic problems of great, and increasing, industrial relevance within many branches of industry and also of great potential value in many other societal areas, such as in complex public management systems and in administrative and other processes. We find that the Programme is an important part of the fulfilment of many of those goals for Swedish and European Industrial Policy and Research Policy which are quoted above as making up those policies. The Programme certainly begins to show results and to promise future results which are in full keeping with the programme planners own ambitious goals for this Programme. Utvärderarna rekommenderade starkt en fortsättning av programmet. I arbetet med planerna för perioden 1997 1999 beslutade NUTEK, bl.a. som följd av utvärderingen, att i anslagsframställan föreslå en fortsättning med betydande utökning av innehåll och budget i ett program kallat Komplexa Tekniska System. Detta förslag övertygade även regeringen om vikten och framgångsmöjligheterna i en sådan satsning. 3. Turbulensen inför och under 1997 När programmet Systemteknik och utvecklingsmetodik för komplexa tekniska system startade, arbetade NUTEK med budgetårsskiften vid 1 juli. På ett tidigt stadium under treårsperioden 1993 1996 bestämde regeringen att man fr.o.m. 1 januari 1997 skulle arbeta med budgetår som överensstämde med kalenderåren. Den förlängning av budgetåret 1995/96 till 18 månader som genomfördes, vållade inga stora bekymmer för programmet inom komplexa system, eftersom programmet startade med planeringsprojekt sommaren 1993 och först från årsskiftet 1993/94 på allvar började dra betydande kostnader. Den förlängda treårsperioden fram till 1 jan 1997 kunde därmed finansieras utan några egentliga omplaneringar. Som framgått ovan planerades en betydande ökning av programmet fr.o.m. 1 januari 1997, men regeringen avsåg också att stiftelserna från de gamla löntagarfonderna, främst då Stiftelsen för strategisk forskning och Stiftelsen för kunskap och kompetens, skulle bidra till delar av NUTEKs forskningsprogram. Förhandlingarna om lämpliga lösningar drog ut på tiden och först i början av hösten 1997 kunde verksamhetsplaneringen för de program, som NUTEK Teknik skulle driva vidare, ta ordentlig fart. Under tiden förlängdes på oförändrad nivå de pågående projekt som förväntades ingå i de framtida programmen, först för första halvåret 1997 och sedan för andra halvåret. Dessa förlängningar avsåg då en övervintringsnivå innebärande bl.a. att inga nya doktorander fick engageras eller andra nya åtaganden fick göras. 4

Appendix 1 (in Swedish)/Bilaga 1 Konsekvenserna av detta blev givetvis mycket stora för projektens planering och engagemang inför framtiden. Att driva en verksamhet på sparlåga och i ovisshet under en tid överstigande ett år innebär givetvis att man kan tappa kompetens, exempelvis bytte doktorander verksamhetsinriktning och därmed hade man bekymmer att sedan komma igång med en accelererande verksamhet. 4. Programmet KTS 1997 2000 Syften och mål Målet för det nya programmet var i grunden detsamma som för det tidigare, men baserat på erfarenheterna från den tidigare programverksamheten formulerades målen mera specifikt: Resultatområdet Komplexa tekniska system inriktar sig mot framtagning av generella metoder, verktyg, beskrivningsmodeller, utvecklingsmetodik mm - för att utveckla, hantera, producera och använda komplexa tekniska system - och samtidigt kunna bemästra de med komplexiteten förknippade svårigheterna. Satsningen syftar till att nå synergieffekter genom att knyta samman kunskaper och kompetens från olika delområden och bygga vidare på satsningar som NUTEK gjort inom olika teknikområden som kan bidra med viktiga kunskaper och hjälpmedel. Satsningen syftar också till att bidra till ökad säkerhet för människa och samhälle vid användning av komplexa tekniska system. Programmets innehåll Som tidigare nämnts skulle det nya programmet både ekonomiskt och innehållsmässigt ha avsevärt större omfattning än det gamla. Det fick då bilda ett eget resultatområde, som från början indelades i fem delområden: - Delområde 1, Systemutveckling, som berör grundläggande och generella frågeställningar kring komplexa tekniska system och ger den metodmässiga grunden. - Delområde 2, Telekommunikationssystem, som tar upp viktiga klasser av tillämpningar med inslag av grundläggande metodfrågor, utveckling av verktyg och hjälpmedel samt utveckling av IT-system inom telekommunikationsområdet. - Delområde 3, Systemarkitektur (Inbyggda system), tar också upp viktiga klasser av tillämpningar med inslag av grundläggande metodfrågor, utveckling av verktyg och hjälpmedel samt utveckling av IT-system för inbyggda datorsystem. 5

Appendix 1 (in Swedish)/Bilaga 1 - Delområde 4, Ledningssystem för komplexa operationer, är ytterligare ett viktigt tillämpningsområde med inslag av grundläggande metodfrågor. - Delområde 5, Modellering och simulering, är inriktat på tillämpning och vidareutveckling av viktiga verktyg och hjälpmedel i olika faser av utvecklingsarbete och användning av komplexa tekniska system. Dessa delområden inkluderade också fortsättningarna av ett antal sedan tidigare pågående satsningar inom bl.a. Software Engineering, Realtidssystem och Reglerteknik. Under programmets gång utökades det sedan från och med 1999 med Delområde 6, Optimering av Infrastrukturer, som är inriktat dels på optimering ur resurssynpunkt av stora tekniska infrastrukturer med distribuerad kontrollstruktur, dels på utformningen av dessa strukturers drifts- och hanteringssystem. Från 1999 kompletterades också programmet med en inriktning mot sensordatafusion, som placerades inom delområde 4. Programmets organisation Inom NUTEK organiserades arbetet som en intern matris med en sammanhållande matris/program/planeringsansvarig. Den övergripande styrningen och samordningen hanterades av en styrgrupp, vars uppgift var att bl.a. beakta helhetsaspekter och integration mellan de olika delområdena. Varje delområde eller sammanhållet projektområde (kluster) hanterades operationellt av en beredningsgrupp eller av en kluster-koordinator. I beredningsgruppen och klustret ingick förutom en eller två representanter för styrgruppen en NUTEK-handläggare, som internt ansvarade för arbetet inom gruppen/klustret. Av historiska skäl hanterades delområdet Telekommunikationssystem av en separat styrgrupp, vars bedömningar inte överprövades av styrgruppen för Komplexa Tekniska System (KTS). Som tidigare nämnts planerades KTS som ett eget resultatområde inom NUTEK Teknik. Budgeten för treårsperioden 1997-99 (senare ändrad till 1998-2000) var planerad till 175 Mkr. Resultatområdet krävde givetvis en betydande intern arbetsinsats som skulle levereras av många handläggare från olika enheter. Tyvärr ledde den tidigare omtalade samordningen av den tekniska forskningen vid NUTEK med Stiftelsen för strategisk forskning och Stiftelsen för kunskap och kompetens till en sänkning av personalstyrkan vid NUTEK Teknik till 2/3 vid årsskiftet 1997/98. Följden blev att också KTS skulle minska sina personalinsatser till 2/3 men programmet genomföras enligt plan. En matrisverksamhet är naturligtvis extra känslig för personalneddragningar och programmet kunde inte fullfölja den höga ambitionsnivån i projektuppföljningen mm. 6

Appendix 1 (in Swedish)/Bilaga 1 Programmets genomförande Utvärderingen av det tidigare programmet inom komplexa tekniska system och de samlade erfarenheterna av genomförandet visade att de grundläggande arbetsmetoderna för programmets genomförande fungerade på ett bra sätt. Man kan för komplexa tekniska system inte starta med en generell ansats och med denna som utgångspunkt få fram användbara metoder. I stället måste man utgå från bearbetning av ett eller flera tillämpningsexempel och sedan generalisera metoder och metodik till andra tillämpningsområden. Den stora skillnaden jämfört med det tidigare programmet var omfattningen, både vad gäller innehåll och mängden projekt. Styrgruppen ansågs ha kapacitet att hantera det utökade programmet och fick därför förtroende att fortsätta sitt arbete med några smärre förnyelser främst betingade av organisatoriska förändringar. Den tidigare omnämnda indelningen av resultatområdet i ett antal delområden skulle möjliggöra att man inom den övergripande ramen bättre kunde uppnå fokuseringseffekter och resultat i forskningsfronten. För att ytterligare förbättra den möjligheten samlades relaterade projekt i kluster där samverkan ytterligare kunde förstärkas. Resultat inom varje kluster eller delområde skulle sedan i möjligaste mån generaliseras för att kunna utvärderas/användas inom andra områden. På grund av den tidigare nämnda samordningsturbulensen gjordes första utlysningen av ansökningsmöjlighet till programmet först hösten 1997, dvs avsevärt försenat i förhållande till den ursprungliga programplaneringen. Ansökningarna skulle var inlämnade den 10 november, sedan vidtog ett ingående bedömningsarbete utifrån de kriterier som styrgruppen beslutat om. I bedömningsarbetet hade beredningsgrupperna kompletterade med externa experter en stor arbetsuppgift. I december fattade styrgruppen beslut om att rekommenderar stöd till 56 projekt med en programkostnad på 46 Mkr för 1998. Av besluten framgick också att programmet skulle ha en gemensam "Kick-off" den 21 april. Vid detta tillfälle skulle då villkor som fanns i besluten vara uppfyllda och avtal om den industriella finansieringen och det långsiktiga samarbetet vara skrivna. Totalt sett var styrgruppen inte helt nöjd med inriktningen av de projektförslag som inkommit. Fortfarande ansågs processindustrin för dåligt representerad i programmet och nya försök att aktivera denna industrisektor gjordes genom speciella utredningar. För detta ändamål men också för att kunna göra anpassningar till utvecklingen inom olika områden reserverade styrgruppen medel för nya ansökningsomgångar. Hösten 1998 genomfördes sålunda i samband med behandlingen av fortsättningsansökningar en ny ansökningsomgång med en budget på cirka 10 Mkr och motsvarande utlysning planerades för hösten 1999. Nya förändringar av forskningsfinansieringen och dess organisation stundade dock och som beskrivs närmare nedan hamnade programmet under hösten 1999 och under år 2000 i akuta bekymmer, som medförde att utlysningen 1999 fick inhiberas. 7

Appendix 1 (in Swedish)/Bilaga 1 Budgetneddragning för år 2000 Budgetplaneringen för programmet hade tagit sikte på att hela treårsperioden 1998 2000 skulle hållas inom en given ram (cirka 175 Mkr), där upptrappning och nedtrappning skulle kunna ske med ett ansvarstagande för de projekt som skulle fortsätta från tidigare program. Upptrappningen innebar då dels att ett antal projekt påbörjades 1 januari 1998 med avsikt att de skulle pågå till utgången av år 2000 och dels att ett antal projekt beslutades först under hösten 1998 men med avsikt att pågå t.o.m. utgången av 2001. Våren 1999 kom dock nya direktiv från regeringskansliet som komplicerade situationen. Riksdagen fokuserade budgetprocessen på ett utgiftstak som inte fick överskridas och i princip försvann de tidigare reservationsanslagen för exempelvis NUTEKs forskningsprogram. Konsekvensen blev att tilldelade medel måste förbrukas det budgetår, som de hade beviljats för, utan någon garanti om att fortsatt stödbehov skulle tillgodoses av statsmakterna i efterföljande budget. Därtill kom den osäkerhet som hörde samman med att olika utredningar tillsattes, som skulle se över hela forskningsorganisationen. Normalt skulle programmet under hösten 1999 ha beviljat medel av 1999 års anslag till projekten för deras fortsatta verksamhet under år 2000. Med hänsyn till direktiven om utgiftstak ansågs detta inte möjligt, utan de medel som programmet reserverat för verksamheten år 2000 användes till en del för att täcka andra NUTEK-utgifter under 1999. Avsikten var sedan att programmet skulle få ersättningsmedel av anslagen för år 2000. För att åstadkomma en bättre integration av de olika verksamheterna inom NUTEK utarbetades emellertid en ny organisation inom NUTEK under senhösten 1999. Denna organisation trädde i kraft 1 januari 2000 och därmed kom helt nya förutsättningar att gälla för medelstilldelningen till programmet för år 2000. Först i mars månad 2000 klarnade budgetsituationen, men bantningen för programmet blev då så betydande att samtliga projekt fick se sitt stöd neddraget med belopp som motsvarade fyra månaders verksamhet. Sett över totala projekttiden 36 månader kan detta synas måttligt, men neddragningen blev mycket abrupt då de projekt som skulle fått medel året ut, fick besked i slutet av mars om att de endast får finansiering t.o.m. augusti. För ett mindre antal projekt som skulle pågå ytterligare ett år blev neddragningstakten mera måttlig. Utan att alla detaljer i processen ovan kunnat belysas, är det ändå uppenbart att problemen för projekten blev svåra man hade avtal med industriföretag om verksamhet för hela år 2000 och i slutet av mars får man besked om att finansieringen upphör sista augusti. Givetvis hade styrgruppen ingående diskussioner om hur situationen bäst skulle lösas en annan lösning, som diskuterades men förkastades på grund av att konsekvenserna ansågs bli ännu värre, var att avbryta ett antal projekt redan i mars månad. Utvärdering Maj 2000 Under hösten 1999 hade styrgruppen beslutat att en utvärdering av programmet skulle ske i maj 2000. Den skulle ta upp projektens vetenskapliga kvalité men också behandla helhetseffekterna av programmet. Utvärderingen med tio stycken utvärderare, varav två var generalister som skulle bedöma den industriella relevansen, genomfördes koncentrerat under en veckas tid. De åtta specialistutvärderarna var engagerade under 2 á 3 dagar, 8

Appendix 1 (in Swedish)/Bilaga 1 medan generalisterna följde arbetet hela veckan. Utvärderingen redovisas i NUTEKrapport R 2000:19. Den vetenskapliga kvalitén ansågs vara god, i vissa fall mycket god. Generalisternas bedömning var mera tveksam. I efterhand kan man möjligen konstatera att programmets totala omfattning var för stor för att kunna greppas på ett bra och tydligt sätt av utvärderarna på den korta tidrymd som de hade till förfogande. Till denna svårighet kommer naturligtvis också den pressade situation som projekten befann sig i när man med den ovan nämnda korta förvarningstiden skulle avrunda dem. Det var säkert inte helt lätt att redovisa vad som åstadkommits på ett rättvisande sätt. 5. Workshop om behov och forskning rörande Komplexa system Okt 2000 Löpande diskuterade styrgruppen förändringar av programmets innehåll och långsiktiga inriktning både vid sina interna möten och vid konferenser och workshop med pågående projekt. Under hösten 1999 påbörjade styrgruppen diskussioner kring hur forskningsbehov och insatsbehov borde marknadsföras framför allt från industrins sida gentemot politiker och organisationer. Med pågående utredningar och omorganisationer och den röriga finansieringsbilden var det dock svårt för styrgruppen att komma till skott målgruppen för en marknadsföringsinsats var kanske alltför diffus. I oktober 2000 genomförde styrgruppen en särskild workshop för att tydligare klarlägga behovet av fortsatta insatser och forskning inom området. I inbjudan till seminariet ingick en sammanställning av styrgruppens tankar om ett framtida systemprogram. En intressant samling Position papers presenterades vid seminariet. I gruppdiskussioner behandlades följande olika aspekter av tänkbara framtida inriktningar: Systeminnovationer och tillämpningar Arkitektur och teknikplattformar Utvecklings- och produktionsmetoder Systemvetenskap/verktyg och metoder Innehåll/avgränsningar/arbetssätt i ett program En sammanställning av resultatet från diskussionerna finns som separat arbetsrapport. Slutsatsen från seminariet utmynnade i att ett program borde innehålla inriktningar som ökade förståelsen för komplexa system, ökade förmågan att ta fram komplexa system samt ett större visionärt tillämpningsexempel. 9

Appendix 1 (in Swedish)/Bilaga 1 Referenser A: Utvärdering av programmet Systemteknik och utvecklingsmetodik för komplexa tekniska system. NUTEK-rapport R 1996:61 B: Verksamhetsplaner. Verksamhetsplan för Tekniska FoU-programmet för 1997 Verksamhetsplan för Resultatområdet Komplexa tekniska system för perioden 1997-1999 (daterad AnH/1997-04-28) Verksamhetsplan för Resultatområdet Komplexa tekniska system för 1998 Verksamhetsplan för Resultatområdet Komplexa tekniska system för 1999 Verksamhetsplan för Resultatområdet Komplexa tekniska system för 2000 C: Utvärdering av programmet Komplexa tekniska system i Maj 2000. NUTEK-rapport R 2000:19. D: Inbjudan till workshop den 31 oktober 2000 inkluderande en sammanställning av styrgruppens tankar om ett framtida systemprogram. E: Position papers presenterade vid workshop den 31 oktober 2000 F: Sammanställning av workshop den 31 oktober G: Samling av dokument rörande styrgruppens diskussioner rörande framtida program Nov 2000 aug 2001 10

Appendix 2 (in Swedish)/Bilaga 2 Styrgruppens metoder för verksamhetsstyrning och intervenering 1. Policy och övergripande verksamhetsstyrning Styrgruppens egna behovs- mål- och policydiskussioner i samverkan med handläggargruppen Kontinuerligt har styrgruppen vid sina möten diskuterat behov, mål och policy. Dessa diskussioner har förts tillsammans med den adjungerade gruppen av NUTEK-handläggare och kontaktpersoner för enheterna. Detta har haft ett dubbelt syfte. Dels har styrgruppen på detta sätt fått policyinformation från olika program- och disciplinområden på NUTEK och dels har man kunnat använda handläggargruppen som ett medel att föra ut sina egna bedömningar till det mycket stora antalet projekt och aktörer inom programmet Komplexa Tekniska System. Bland de övergripande frågeställningar, som har varit uppe till diskussion, är avgränsningen av begreppet komplexa tekniska system. Själva rubriceringen avgränsar området till teknikområdet, varför stora ekonomiska system och allmänna samhällssystem kunde uteslutas. Styrgruppens olika ledamöter har naturligtvis med bakgrund från sina egna ämnesområden och erfarenheter haft åsikter om avgränsningen, men man kunde relativt lätt enas om att begreppet förutsatte aggregat som bestod av delar som kunde arbeta med olika tekniska medier och som förutsatte kunskaper från flera discipliner. Helheten ansågs vara större än summan av delarna. Utifrån dessa synsätt fann man också att det var skillnad på komplexa problem och komplicerade problem. Exempelvis var man enig om att hydrauliska strömningsproblem, flygtekniska fladderproblem, mekaniska plåtformningsproblem eller kemiska reaktionsprocesser kunde vara mycket komplicerade och svåra att lösa, men att som isolerade företeelser studera denna typ av komplicerade problem ansågs inte höra hemma inom området komplexa tekniska system. Fokus ansågs också vara att hantera komplexiteten på ett sådant sätt att systemen blev lättanvända och gripbara, trots att de kunde vara funktionellt avancerade och internt mycket komplexa. Viktigt var också att programmet var metodinriktat vilket innebar att de metoder och lösningar som utvecklades skulle kunna generaliseras så att de kunde appliceras inom andra tillämpningsområden och applikationer. I slutet av 1999 och år 2000 stod frågorna rörande programmets fortsättning och den mera kortsiktiga finansieringsproblematiken för programmets avslutning ständigt på styrgruppens dagordning. Långsiktigt hade styrgruppen många idéer om hur ett kommande program skulle se ut för att tillvarata de unika svenska kvaliteterna inom området och hur dessa förslag skulle marknadsföras för att få effekt. Många gånger ledde ordföranden mycket stimulerande 1

Appendix 2 (in Swedish)/Bilaga 2 diskussioner och skrev blädderblocksbladet fullt med alla inspel för att avsluta med att dra upp riktlinjerna för en struktur. Syftet med ett fortsatt program skulle vara att stärka svensk industris systemförmåga genom att göra den komplexa verkligheten förståelig och hanterbar i olika mänskliga roller. Styrgruppen hade uttalad uppbackning från de stora företagens ledningar men lyckades i den turbulenta omorganisationen av den statliga forskningsfinansieringen inte identifiera målgruppen för den marknadsföring som man ville genomföra. På kortare sikt diskuterades olika lösningar för finansieringen av programmets avslutning under 2000 och 2001. Styrgruppen efterlyste intensivt en konsekvensanalys av budgetförändringarna och konstaterade att den situation som åstadkommits äventyrade ett viktigt område för svensk industri och saboterade de väsentliga initiativ som tagits med programmet. Budgetbesluten kunde få stora konsekvenser för de inblandade småföretagen och deras kontakter med högskola och stora företag, liksom för doktorandernas industriella kontakter. En av de mera konkreta frågeställningar som styrgruppen engagerade sig i, var de inbördes avtalen mellan parterna i respektive projekt. Här ansåg speciellt industrirepresentanterna i gruppen att det var viktigt med tydliga skrivningar om rätten till resultaten och att avtalsfrågan skulle föras upp till hög nivå hos parterna för att samarbetet skulle få tillräcklig prioritet. Styrgruppen poängterade också att avtalen skulle vara långsiktiga och täcka projektets hela löptid. Dessa frågeställningar gjorde att avtalsskrivningen för vissa projekt drog ut på tiden och styrgruppen fick till en del backa på långsiktigheten med hänsyn till budgetprocesserna och acceptera att avtalen var årsvisa vad gällde ekonomin. Programmets innehåll via indelning i delprogram och deras innehåll Som har framgått av den separata historiska beskrivningen av NUTEKs program inom området komplexa tekniska system gjordes en kraftig utökning av programmet 1997. I samband med denna utvidgning gjordes också en indelning av programmet Komplexa Tekniska System i delområden för att underlätta kontakterna mellan projekt med likartade frågeställningar. För att ytterligare förstärka kontaktmöjligheterna arbetade styrgruppen med att införa speciella projektgrupperingar kluster där någon handläggare eller projektledare hade ett speciellt ansvar för koordinering och för att ordna gemensamma diskussioner inom projektområdet. I stor utsträckning definierades dessa kluster före programmets början med utgångspunkt från speciella utredningar och åtföljande seminarier. Detta var då utredningar och beslut som initierades av styrgruppen för det första programmet inom området komplexa tekniska system, men genom att större delen av denna styrgrupp engagerades i det nya programmet var det aldrig några problem med kontinuiteten. Som framgått var programmet omfattande och ett stort antal delområden och kluster definierades. För att operationellt kunna hantera dessa med tillräcklig områdeskunskap och överblick utsågs inom styrgruppen särskilda beredningsgrupper, som ansvarade för respektive delområde/kluster och som bemannades med styrgruppsledamöter och handläggare. 2

Appendix 2 (in Swedish)/Bilaga 2 När första ansökningsomgången genomfördes hösten 1997 konstaterade styrgruppen att man saknade vissa tillämpningsområden bland ansökningarna och gärna hade sett flera industridrivna projekt. Styrgruppen höll därför inne en del medel för att kunna organisera särskilda projekt som tillgodosåg dessa önskemål och kunna arbeta proaktivt mot industrin. Huvudparten av resterande medel fördelades av styrgruppen på delområden och kluster efter ansökningarnas kvalité med viss hänsyn till ansökningstryck. Styrgruppen diskuterade återkommande de industriella behoven inom området och utnyttjade under programmets gång möjligheterna till att skapa en lite friare struktur genom att definiera helt nya delområden och samordna projektverksamheter i kluster. Exempel på större sådana områden som infördes är Optimering av infrastrukturer och Sensordatafusion. Andra nya områden som styrgruppen diskuterade men som inte infördes som sammanhållna kluster utan enbart poängterades som angelägna i programdokumenten vid utlysning av ansökningsomgångarna var Adaptiva systemarkitekturer, Distribuerad interaktiv simulering och Produktens livscykel. Programspecifik kriterielista Vid sidan av de allmänna projektbedömningskriterierna om industriell relevans och ekonomisk potential, vetenskaplig kvalité och kompetens samt projektuppläggning diskuterade styrgruppen fram ett stort antal programspecifika kriterier som var viktiga för programmets genomförande och den industriella användningen av de metoder och resultat som togs fram. Utöver dessa kriterier tog styrgruppen också fram bedömningskriterier som avsåg de speciella delområdena, exempelvis gällde för klustren att förslag prioriterades om de hade goda samverkansmöjligheter med andra projekt, och de tillsammans med dessa bidrog till att programintentionerna uppfylldes. Även här skedde successiva fokusförändringar för att ta till vara de erfarenheter om brister som kom fram i projektarbetet och programarbetet. Styrgruppen använde sig härvid i större utsträckning av morötter i form av bonusmedel än piska för att uppnå de förändringar som man önskade. Exempel på sådana förändringar som man satte hög prioritet på var: att många projekt inte arbetade med human factors i den utsträckning som styrgruppen önskade (speciella medel för riktade insatser infördes) att den traditionella processindustrin deltog i alltför begränsad utsträckning (speciella medel för riktade insatser infördes samt speciella utredningar och informationsinsatser genomfördes) att den industriella användningen skulle beaktas i större utsträckning vid projektbedömningen 3

Appendix 2 (in Swedish)/Bilaga 2 2. Styrning av projekturvalet och projektinnehållet Särskilda bedömningsgrupper med styrgruppsledamöter, externa experter och handläggare bemannas För att kunna behandla det stora antalet ansökningar och projekt som ingick i programmet Komplexa Tekniska System fördelade styrgruppen arbetet för olika delområden och kluster på de beredningsgrupper som omnämnts tidigare. De styrgruppsledamöter och handläggare som ingick i dessa kompletterades då med externa experter med specifik kunskap inom delområdet. Utgångspunkten för bedömningsarbetet var de bedömningskriterier som styrgruppen hade diskuterat fram. I beredningsgruppens uppgift ingick inte enbart att framföra rekommendationer till styrgruppen utan beredningsgruppen hade en långsiktigare uppgift att följa de beslutade projekten över tiden och komma med nya rekommendationer i samband med de årliga omprövningarna av projekten. Detta innebar då att beredningsgruppen kunde föra fram innehållsrekommendationer till styrgruppen och projektet och vid omprövningen beakta hur man hade lyckats följa de planer och förväntningar som man hade presenterat i ansökan. Användning av beredningsgrupper hindrade inte på något sätt styrgruppen från att ha mycket djupgående diskussioner om vissa projekt, deras innehåll och betydelse för programmet i stort. Genom integrationen av styrgrupp och beredningsgrupp kunde man hela tiden ha en öppen dialog. Trots den relativt hårda styrningen mot tekniska system som var mera direkt relevanta för industrin, lämnades också utrymme för framtidsteknik, exempelvis ett projekt, som arbetade med genetisk programmering tillämpad på kommunikationsväxlar och humanoida robotar. Mera näraliggande fanns också ett projekt som behandlade servicerobotar och autonoma system. Resultatuppföljning. Som tidigare nämnts ingick styrgruppsledamöterna i de beredningsgrupper som hade uppföljningsansvar. Därutöver var styrgruppsledamöterna faddrar för vissa projekt och kunde då adjungeras till projektstyrgruppen och kunde därmed mera konkret följa vad som hände i projekten. Speciellt de stora projekten följdes på detta sätt upp kontinuerligt för de mindre blev det mera uppföljning i kritiska skeden. Grunden för denna uppföljning utgjordes av ansökningarna, där projekten av styrgruppen uppmanats att detaljerat redovisa sina ekonomiska planer och planerade etappmål. Styrgruppen intresserade sig också starkt för den formella uppföljningen av projekten och hade med utgångspunkt från rapporteringsrutiner i industrin tagit fram förslag till projektuppföljning och förslag till särskilda rapporteringsblanketter som regelbundet skulle skickas in. Disciplin i projektens rapportering kunde åstadkommas genom att utbetalningen av medel hölls inne, tills avsedd rapportering hade lämnats in. Till följd av den ekonomiska turbulens och de personalneddragningar som drabbade programmet i slutskedet och som 4

Appendix 2 (in Swedish)/Bilaga 2 beskrivits i historiken kunde rapporteringsuppföljningen inte genomföras fullt ut på det sätt som styrgruppen önskade sig. Exempel på styrgruppens konkreta ingripanden i projekt och deras förutsättningar Nedan redovisas några exempel, som visar några olika slag av mer eller mindre direkta ingripanden som styrgrupp och programledning har gjort för att förbättra förutsättningarna för projekten samt utveckla dem och deras samverkansmöjligheter. Ex. 1: Inom ett projekt arbetade man med statistiska metoder för styrning av fjärrvärmeproduktion och -distribution. Projektet hade starka kopplingar till energiteknikutveckling och under den period då Energimyndigheten tillhörde NUTEK samfinansierades projektet av Komplexa tekniska system och Energiprogramsidan. När Energimyndigheten blev en självständig myndighet försvann samfinansieringen men styrgruppen ansåg att projektet hade större kontaktytor och fler synergimöjligheter med energiprogrammen och arbetade därför under fortsatt finansiering tålmodigt med en överflyttning till Energimyndigheten. Så småningom kom också en lösning till stånd med finansiering denna väg. Ex. 2: Syftet med ett annat projekt var att utveckla nya arkitekturer för modellering och styrning av flexibla tillverkningssystem. Genom att projektet hade anknytning till flera industrisektorer såsom styrsystemområdet, verkstadsindustrin och processindustrin hade man tidvis svårigheter att uppfylla styrgruppens krav på att presentera samarbetsavtal inom stipulerade tidsramar och också problem med att få avtalen tillräckligt långsiktiga. Styrgruppen har därför dels vid några tillfällen gett projektet dispens från angivna deadlines men också via personliga ingripanden av ledamöter medverkat till att lösa problemen. Projektet hade också finansiering från andra program, vilket i kombination med de många samarbetsparterna gjorde att styrgruppen fann det nödvändigt att ställa krav på en tydlig särredovisning av den industriella medfinansieringen. Ex. 3: Inom lednings- och beslutsstödsystem behandlade ett projekt informationsförsörjning och systematiskt återbruk av nyttigheter som framkommit vid utvecklingsarbete. Projektet ansågs idémässigt intressant men diskuterades i flera omgångar av styrgruppen på grund av att det tidvis var alltför konsultdrivet med stora brister i medfinansiering och forskarmedverkan. Först efter betydande personliga förändringsinsatser från styrgruppsledamöter och beredningsgrupp fick projektet sådan stadga och forskarmedverkan att styrgruppen tillstyrkte den fortsatta finansieringen. Ex. 4: I ett omfattande och långsiktigt projekt knöt man ihop flera verksamheter på distans vid flera tekniska högskolor och vid flera olika företag. Genom styrgruppens och programledningens positiva stöd kunde denna distansverksamhet utgöra tillämpning för ett särskilt finansierat NUTEK-projekt rörande konstruktionsarbete på distans över bredbandsnätet. Liksom i flera andra dynamiska projekt hade styrgruppen en svår balansgång mellan den positiva synen på projektets intressanta idémässiga innehåll och motiverade kritiska synpunkter på projektets tidplaner, mål och måluppfyllelse. Genom en öppen dialog mellan styrgrupp och projektledning kunde man få en uppstyrning utan att alltför mycket störa de idémässiga kvaliteterna. 5

Appendix 2 (in Swedish)/Bilaga 2 Ex. 5: En inriktning mot teamwork fanns i två projekt som båda arbetade med ramverk och metodik för konstruktion och hantering av komplexitet, båda främst med stora företag som samarbetsparter. Projekten handlade i högsta grad om de mjuka delarna i projektarbetet och mindre om de tekniska problemen. Styrgruppen har vid konferenser, seminarier och presentationer ofta aktivt lyft fram projekten och poängterat resultatens betydelse för systemarbetet. Ex. 6: Nästa exempel är ett äldre projekt som arbetade med framtagning av programverktyg för systemutveckling och som finansierades av det ursprungliga programmet inom komplexa system. Projektet drevs av ett litet företag och fick med styrgruppens positiva stöd också finansiering från NUTEKs småföretagsprogram. Styrgruppen värderade också fortsättningsvis projektet positivt och medverkade till marknadsföring och användning av verktyget via särskilda stödpengar till de projekt inom programmet som ville använda verktyget. Ex. 7: I ett projekt inom systemutveckling gick styrgruppen utanför det vanliga fältet för komplexa tekniska system genom att stödja forskare vid Karolinska institutet i samverkan med ett företag som arbetar med genetisk diagnostik. En grundförutsättning för projektet var då företagets egen informationstekniska kompetens. Efter diskussioner och krav från styrgruppen kunde projektet utökas med datavetenskapliga doktorander. Ett annat krav som styrgruppen ställde var att man skulle titta på möjligheterna att överföra resultaten till andra tillämpningsområden. På grund av den lite plötsliga nedtrappningen av programmets verksamhet blev denna uppgift lite styvmoderligt behandlad. Utvärdering av vetenskaplig kvalitet och relevans I planeringen av utvärderingen i Maj 2000 av programmets verksamhet tryckte styrgruppen kraftigt på att programmets industriella relevans måste bedömas inte bara projektens vetenskapliga kvalité. Relevansbedömningen ansågs nämligen vara mycket viktig för beslut om programmets eventuella fortsättning. Med utvärderingsrapporten i sin hand konstaterade styrgruppen att generalisternas sammanfattning och slutsatser av fyra dagars bevakning av projektutvärderingen var innehållsmässigt mager och en besvikelse. Ledamöter i styrgruppen konstaterade att utvärderarnas syn på komplexitet var en annan än styrgruppens och inte så välutvecklad. Styrgruppen konstaterade att projektens vetenskapliga kvalité ansågs vara hög men att specialistutvärderarna i vissa fall hade påpekat att projekt arbetat på en något för elementär nivå. State of the art måste vara utgångspunkten och beskrivas tydligare. Detta ansåg styrgruppen var viktigt att ta fasta på och arbeta vidare med. Även om generalistutvärderarna vacklade mellan de två synsätten att fokusera starkt på några få tillämpningsområden och att mera övergripande behandla komplexitetsproblematiken för hela fältet, så såg styrgruppen ingen anledning till att gå tillbaks till den gamla indelningen med separata program, utan ansåg att en sammanhållning av flera områden i ett program kan ge stimulans och synergieffekter. 6

Appendix 2 (in Swedish)/Bilaga 2 Klustertanken är praktiskt svår att genomföra, eftersom många projektledare har fullt upp med sitt eget område där de i första hand skaffar sina akademiska meriter. Detta kan förklara utvärderarnas bedömning att klustermetoden ej varit effektiv. Styrgruppen ansåg dock att metoden successivt skulle bli effektivare och förstärkas när de horisontella projekten fick växa till sig. 3. Resultatspridning, generalisering och resultatanvändning Informations- och Kickoff-konferenser I samband med utlysning av ansökningsomgångarna har programmet anordnat särskilda informationsdagar där programplaner, projektkrav och andra villkor presenterats, ofta tillsammans med intressanta projekterfarenheter och framtidsvisioner. När ett större antal projekt har startats, exempelvis våren 1998 och 1999 har styrgruppen funnit det lämpligt att göra särskilda sammandragningar av samtliga projekt för att medvetandegöra alla om den totala verksamhetens innehåll och därmed vilka potentiella samarbetsparter som kan finnas för de enskilda projekten. Särskilt fokuserades då på vilka projekt som ingick i det delområde eller kluster som man tillhörde. Ett annat syfte med programmets kickoff-konferenser var att poängtera att nu skall alla förberedelser för projektarbetet, såsom avtal mellan aktörerna, detaljerad uppföljningsbar projektplan mm, vara klara. Beredningsgrupperna hade inför dessa kickoff-konferenser till uppgift att fokusera på vad som saknades i projekten och deras planer. Liknande sammandragningar men med något annorlunda fokus gjordes när krissituationer uppstod såsom i samband med neddragningarna år 2000. Härvid var fokus på information om situationen och tips om åtgärder rörande hur det enskilda projektet kunde hantera sin situation. Programkonferenser och workshops med särskilda arbetsteman Årligen har programmet genomfört större programkonferenser och därutöver särskilda workshops för att diskutera exempelvis framtida behov och programinriktning. Vid programkonferenserna har olika organisationsformer prövats för att finna synergier och samarbetsmöjligheter mellan projekten men också för att hitta generaliseringsmöjligheter och överföringsmöjligheter till andra tillämpningsområden för framtagna metoder och resultat. Vid sidan av plenumföredragningar av projekt som har bedömts ha mera allmänt intresse samt postersessioner och gruppredovisningar inom delområdena och klustren har också gruppsammandragningar med indelning efter särskilda teman genomförts. Den senare metoden har varit framgångsrik för att finna nya konstellationer av projektsamverkan men har krävt förhållandevis stor projektkännedom och arbetsinsats för konferensorganisationen. Frågorna kring programmets fortsatta inriktning fanns alltid med på dagordningen vid olika konferenser. I oktober år 2000 organiserade styrgruppen dessutom ett större specialseminarium, där ett stort antal Position papers behandlande framtida system- 7

Appendix 2 (in Swedish)/Bilaga 2 program presenterades. De engagerade diskussioner som fördes vid seminariet var sedan underlag för styrgruppens egna fortsatta diskussioner och slutsatser om vad som borde göras efter programmets avslutning. Horisontella projekt I senare hälften av programmet har styrgruppen i stor utsträckning fokuserat på horisontella projekt. Syftet har dels varit att sammanställa resultat från delområden och kluster i bokform eller liknande för att sprida kunskapen om resultaten och dels att finna generaliseringsmöjligheter för framtagna metoder och möjligheter att överföra metoderna till andra tillämpningsområden. Exempel på projekt av det första slaget är projektet Editing of a book in the Software Engineering Cluster, där projektledarna i klustret via presentationsarbetet dels överför erfarenheter sinsemellan men också bearbetar resultaten så att de kan tillämpas på ett rationellt sätt. Projektet Systems Engineering of Complex Systems är ett exempel av det andra slaget. Här studerades övergripande 12 projekt inom programmet och hur deras metoder skulle kunna appliceras inom olika tillämpningsområden. Parallellt påbörjades likartade horisontella projekt där inriktningen var mera mot ett specifikt tillämpningsområde. Till följd av de stora budgetneddragningar som programmet tvingades till år 2000 blev dessa projekt delvis en missräkning för styrgruppen, eftersom projekten inte kunde fullföljas i den utsträckning som var önskvärd. Den här formen av projekt organiserades dels genom öppna utlysningar av ansökningsmöjligheter men också genom en ackvisition av aktörer och projekt som kunde ge intressanta resultat och sammanställningar. Även på ett tidigt stadium i programmet arbetade styrgruppen med horisontella aktiviteter för att sprida metoder. Ett sådant exempel är Tofs Tool Support for Complex Systems, där programmet subventionerade inköp av licenser och konsultstöd för att andra projekt skulle kunna utnyttja metoden. Programmet kunde också dra fördel av att projektet under sin utvecklingsfas kunde få stöd från NUTEKs småföretagsprogram. Samtidigt visade det sig att företagets litenhet blev ett problem när det gällde spridningen och en bredare användning av metodiken. En avancerat metod ansågs av många kräva ett större företag med en stabil grund för att man skulle våga investera i den kunskapsutveckling och systemutveckling som krävdes för metodens införande. 8

Appendix 3/Bilaga 3 Statistics from the programme From the start of the second phase of the programme in 1997 new routines for supervision were adopted. These were intended to facilitate the possibilities of the steering group to get information on the progress of the projects and for the management to put down figures both for finance control and result reporting while at the same time make life a little bit easier for the project leaders in what has always been looked upon as a burden, namely reporting back all this information. Thus a pair of forms was prepared, one progress report including short-hand notations for progress and results, one for economic follow-up. These were to be checked against a project summary and a result budget (what results when). This was a two-sided form prepared before the project was started and updated at major revisions. Progress was measured simply by statements that this plan was indeed followed (or not followed and why). Result figures were given both for the period reported and aggregated since start. The entries given were not qualified as to a precise meaning, which can be seen as giving room for ambiguity, but nevertheless reflects the project leaders view. The economic form was of course a necessity and in addition gave the important information on the companies contribution, which was of prime interest. The reports were scheduled one per half year and triggered payment for the period as well as, if requested, advance payment for the next. Given the possibility of advance payment and to counterfeit the possibility of a non-show of reports, 10% (of the total aggregate) was withheld until a slutredovisning (final report) was received. Unfortunetly, in the transition from NUTEK to VINNOVA, this condition had to be given up for more macroeconomic reasons, and all projects were paid in advance. This, which was anticipated by the management, resulted in non-show of some of the reports. Primarily because of that, and despite considerable efforts to collect the statistics, not all projects can be included in the programme aggregate statistics. In fact, of 64 projects only 51 (56 for some statistics) have been included (since we did not want to guess) of these however 5 were projects taken over from an earlier programme which did not have the same conditions so statistics is missing from a merely 8 (3) projects. For the future better IT-support should be available for monitoring follow-up activities. Information collected from the projects has been organised in a database. It can be structured in many ways and statistics given i e per area. The statistics given in the following slides is aggregate statistics from the programme. 1

Komplexa Tekniska System Resultatastatistik 1997-2001 Komplexa Tekniska System Resultatstatistik 1997-2001 Historik Komplexa Tekniska System Resultatastatistik 1997-2001 1991IT2000 (Näringsdep Ds 1991:63) 1992 NUTEK-IVA seminarier om Komplexa System 1994 Programmet Systemteknik och Utvecklingsmetodik för Komplexa System ( t o m 96) 1996 Utvärdering 1997 Programmet Komplexa Tekniska System 2001 Programmet avslutas 2002 Utvärdering 1

Komplexa Tekniska System Resultatastatistik 1997-2001 Projekttyp FoU-projekt Övertagna FoU-projekt* Förstudie Avbrutna Horisontella Administrativa (utvärderingar, konferenser) Antal 64 5 1 5 8 3 Inkl stat 51 0 0 0 0 0 Exkl stat 13 5 1 5 8 3 Programmedel 141907 2170 100 6709 2161 916 Summa 86 51 35 153567 * från Inbyggda system Komplexa Tekniska System Resultatastatistik 1997-2001 Programområde Systemutveckling allmänt IT-integritet Software Engineering Produktdatahantering Systemarkitektur* Ledningssystem Sensordatafusion Modellering & Simulering Optimering av infrastruktur Antal FoUprojekt 13 5 8 4 15 5 3 12 4 4 1 2 1 Antal Förstudier 1 Antal avbrutna 2 1 1 1 Antal Horisontella Programmedel (Ksek) 29541 13224 15803 11616 28527 16071 3038 29440 5787 Summa Inkl 5 från Inbyggda system 69 8 1 5 Administrativa uppdrag 144077 2161 100 6709 153047 520 153567 2

Komplexa Tekniska System Resultatastatistik 1997-2001 Fördelning av medel på olika områden 18% 19% 2% 4% 8% 19% 9% 11% 10% Systemutveckling allm Modellering & Simulering Systemarkitektur Ledningssystem Software Eng IT-integritet Produktdatahantering Optimering av infrastruktur Sensordatafusion Komplexa Tekniska System Resultatastatistik 1997-2001 Företagens bidrag Värden baserade på 51 projekt fullständigt rapporterade FoU-projekt MSek 300 250 200 150 100 50 0 S t a t l i g t K o n t a n t N a t u r a S u m m a F ö r e t a g Företagen T o t a l t 100 50 0 99 % Utnyttjade statliga medel % Företagens samfinansiering 51,19 3

Komplexa Tekniska System Resultatastatistik 1997-2001 Företagens bidrag Värden baserade på hela programmet inkl ofullständigt rapporterade FoU-projekt samt 100% finansierade horisontella projekt & administration MSek 300 250 200 150 100 50 0 S t a t l i g t K o n t a n t N a t u r a S u m m a F ö r e t a g Företagen T o t a l t 100 50 0 99 % Utnyttjade statliga medel % Företagens samfinansiering 44,87 Komplexa Tekniska System Resultatastatistik 1997-2001 Deltagare Värden baserade på 56 projekt. Räknat i antal deltaganden (en deltagare kan förekomma i flera projekt). 180 160 140 120 100 80 60 40 20 0 173 80 36 Totalt S to rfö retag SMF 4

Komplexa Tekniska System Resultatastatistik 1997-2001 Akademiska resultat I Värden baserade på 51 projekt 40 38 35 30 25 24 20 15 10 5 0 Doktorsex Licexamen Öst Komplexa Tekniska System Resultatastatistik 1997-2001 Akademiska resultat II Värden baserade på 51 projekt 250 209 200 150 100 86 50 18 0 Rapporter Artiklar Böcker 5

Komplexa Tekniska System Resultatastatistik 1997-2001 Akademiska resultat III Värden baserade på 51 projekt 400 350 300 250 200 150 100 50 0 Föredrag Kurser Nationella Internationella Komplexa Tekniska System Resultatastatistik 1997-2001 Industriella resultat Värden baserade på 51 projekt 30 25 20 15 10 5 0 Metoder Prototyper Produkter Processer Patent 6

VINNOVAs publications April 2004 See www.vinnova.se for more information VINNOVA Analysis VA 2004: 1 The Swedish National Innovation System 1970-2003 VINNOVA Analysis VA 2003: 1 Innovationssystemanalys inom flygindustri och luftfart. Förstudie 2 Swedish Biotecknology - scientific publications, patenting and industrial development 4 Svensk sjöfartsnärings innovationssystem - igår, idag och imorgon VINNOVA Analysis VA 2002: 2 Det Svenska Nyföretagandet 1986-1997 förändringar i företagsstrukturer och sysselsättningseffekter. VINNOVA Innovation in Focus VF 2002: 1 Effekter av VINNOVAs föregångares stöd till behovsmotiverad forskning Fyra effektanalyster av insatser under perioden 1975 2000 (for a short version in swedish & english, see VI 2002:7 & VI 2002:8). Only as PDF 2 Stimulating International Technological Collaboration in Small and Medium-Sized Enterprises. A Study of VINNOVA s SMINT Programme. 3 Regional ekonomisk tillväxt i Sverige 1986 2001. En studie av tillväxtens utveckling i Sveriges lokala arbetsmarknader. VINNOVA Forum VFI 2003: 1 Commercialization of Academic Research Results (Innovation policy in Focus) VINNOVA Forum VFI 2002: 1 Betydelsen av innovationssystem: utmaningar för samhället och för politiken (Innovation policy in Focus) 2 Innovationspolitik för Sverige: mål, skäl, problem och åtgärder (Innovation policy in Focus) 3 Teknikparkens roll i det svenska innovationssystemet - historien om kommersialisering av forskningsresultat (Innovation policy in Focus) VINNOVA Information VI 2004 1 Årsredovisning 2003 VINNOVA Information VI 2003: 1 Verksamhet inom Transporter 2 Årsredovisning 2002 3 VINNOVAs activities within Biotechnology 4 The Competence Centres Programme. Third International Evaluation. Group 1 (8 Centres) 5 The Concept of Innovation Journalism and a Programme for Developing it. Only as PDF 6 EUREKA VINNOVA Information VI 2002: 1 Research and innovation for sustainable growth. Replaces VI 2001:2 2 VINNOVAs verksamhet pågående och planerade aktiviteter. Juli 2002. Replaces VI 2001:10 3 Tillväxt i regioner genom dynamiska innovationssystem 4 VINNOVAs årsredovisning 2001 5 IT i verkstadsindustrin. Program för mångvetenskaplig forskning i samverkan industri, högskola och institut 6 Regionala företagskonsortier 1994-2001 7 Effekter 1975 2000. Stöd till behovsmotiverad forskning. Short version of VF 2002:1 8 Impact of R&D during the period 1975-2000. The impact of VINNOVAs predecessors support for needs. English version of VI 2002:7 9 Verksamhet inom BioTeknik. Speciellt framtagen för BioTech Forum och Medicintekniska konferensen oktober 2002. VINNOVA Policy VP 2003: 1 VINNFORSK - VINNOVAs förslag till förbättrad kommersialisering och ökad avkastning i tillväxt på forskningsinvesteringar vid högskolor. HUVUDTEXT. For appendixes see VP 2003:1.1 1.1 VINNFORSK - VINNOVAs förslag till förbättrad kommersialisering och ökad avkastning i tillväxt på forskningsinvesteringar vid högskolor. BILAGOR. For main text see VP 2003:1 2 Behovsmotiverad forskning och effektiva innovationssystem för hållbar tillväxt. VINNOVAs verksamhetsplanering 2003-2007. Replaces VP 2002:1. For english version see VP 2002:4, for full swedish version see VP 2002:3 3 VINNOVAs forskningsstrategi. Strategi för hållbar tillväxt 4 Nationell Innovations- och forskningsstrategi för området Miljödriven teknikutveckling. Only as PDF VINNOVA Policy VP 2002: 1 See VP 2003:2 2 Nationellt inkubatorprogram 3 Behovsmotiverad forskning och effektiva innovationssystem för hållbar tillväxt. En fördjupad version av VINNOVAs verksamhetsplanering 2003-2007. For short swedish version see VP 2002:1, for short english version

see VP 2002:4 4 Effective innovation systems and problem-oriented research for sustainable growth. VINNOVA s strategic plan 2003-2007. For swedish veersion see VP 2002: 1 and 3 5 Nationell strategi för FoU inom området tillämpning av informationsteknik. VINNOVA Report VR 2004: 1 Nya material och produkter från förnyelsebara råvaror. En framtidsbild och vägen dit. Replaces VR 2002:16 2 Nya material och produkter från förnyelsebara råvaror. Short version of VR 2004:1. Replaces VR 2002:22 3 Evaluation of the NUTEK- VINNOVA programme in Complex Technical Ststems 1997-2001 - Utvärdering av ett FoUprogram i Komplexa Tekniska System 1997-2001 VINNOVA Report VR 2003: 1 Fysisk planering i det digitala samhället (Telematik 2004) 2 Kina störst på mobiltelefoni - konsekvenser för omvärlden (Telematik 2006) 3 Framtidens fordon - mötet mellan två mobila världar (Telematik 2006) 4 Efter 11 september 2001: - Kan storebror hejdas? (Telematik 2006) 6 Kunskapskultur och innovation. Innovationssystem kring energirelaterad vägtransportteknologi. Förstudie. Only as PDF 7 Förändrad finansiering av tranportforskningen. Only as PDF 8 Inledande laboratorieförsök - Projekt AIS 32. Delrapport 1. Only as PDF 9 Inledande fältförsök - Projekt AIS 32. Delrapport 2. Only as PDF 10 Hur går det till i verkligheten? Innovationsprocessen utifrån 18 fall 11 Returlogistik - Utveckling av logistiksystem för returgodsflöden. Only as PDF 12 Genusperspektiv på innovationssystem - exemplet svensk musikindustri VINNOVA Report VR 2002: 1 Explorative System-Integrated Technologies EXSITE 2 Rationalitet och etik i samhällsekonomisk analys och Nollvision. Expertseminarium november 2001. Only as PDF 3 Regionala innovationssystem. En fördjupad kunskapsöversikt. Only as PDF 4 Funktionshindrades resmöjligheter. Sammanfattning av senaste årens forskning. CD with all related reports 5 Organisationsövergångar och unika kulturer. Förändringsdynamik och utvecklingsstöd via Växtkraft Mål 4. For short version see VR 2002:21 6 Metanoldrivna bilar i Trollhättan Göteborg. Förstudie. Only as PDF 7 Hållbart arbete i informationssamhället. Slutrapport från projektet Callcenter i utveckling långsiktigt hållbart arbete med kunder på distans 8 Knowledge exchange, communication and context in electronic networks (KnowHow). Only as PDF 9 Systemiskt lärande som ansats i logistikutvecklingen en studie av internethandeln. Only as PDF 10 Framväxten av en ny vetenskapsbaserad basteknologi (nanoteknik) och dess relevans för det transport-teknologiska området. Förstudie. Only as PDF 11 Den nya ekonomin ett internetperspektiv (Telematik 2004). For short version see VR 2002:12 12 Den nya ekonomin ett internetperspektiv (Telematik 2004). Short version of VR 2002:11 13 Projekt Camelot. Rundabordssamtal och seminarier kring framtidens boende (Telematik 2004). Only as PDF 14 Tyskland och användningen av Internet - en jämförelse med Sverige (Telematik 2004) 15 DIGITALA NYHETER. Nyhetsförmedling via Internet (Telematik 2004). Only as PDF 16 Nya material och produkter från förnyelsebara råvaror. En framtidsbild och vägen dit. For short version see VR 2002:22 17 Transportinformatik och personlig integritet. Only as PDF 18 Utvecklade leverantör kundrelationer: Supply Link Management. Only as PDF 19 Trämekanisk framsyn. Ett projekt för utveckling av den trämekaniska industrin. Slutrapport. Only as PDF 21 En sammanfattning av boken: Organisationsövergångar och unika kulturer. Förändringsdynamik och utvecklingsstöd via Växtkraft Mål 4. Short version of VR 2002:5 22 Nya material och produkter från förnyelsebara råvaror. Short version of VR 2002:16 23 Transporteffektivisering med integrerad informationsteknologi, TRANSMIT. Only as PDF 24 Trä-, Bygg- och Möbelprogrammet - en analys av insatser och resultat 25 Face synthesis as a communication aid for hard-of-hearing people. Teleface l and ll. Final project report. Only as PDF 26 Communication & Services in Open Networks. Kommunikation & Tjänster i Öppna Nätverk. 1999-2002. Only as PDF 27 Utvärdering av teknik som reducerar kväveoxider på äldre arbetsmaskiner genom Selective Catalytic Reduction - SCR. Only as PDF 28 The North European Maritime Container Feeder Market. Only as PDF 29 VinnEr En samverkanspilot mellan VINNOVA och Ericsson. 30 Dialogprojektet - Framtida handel. Rapporter framtagna av Arbetsgruppen för samordning av dagligvarutransporter. Only as PDF

Om/About VINNOVA VINNOVAs uppgift är att främja hållbar tillväxt genom utveckling av effektiva innovationssystem och finansiering av behovsmotiverad forskning. Genom sitt arbete ska VINNOVA tydligt bidra till att Sverige utvecklas till ett ledande tillväxtland. I serien VINNOVA Rapport publiceras externt framtagna rapporter, delrapporter, kunskapssammanställningar, synteser, översikter och strategiskt viktiga arbeten från program och projekt som fått anslag av VINNOVA. Forskning och innovation för hållbar tillväxt VINNOVA s mission is to promote sustainable growth by developing effective innovation systems and funding problem-oriented research. Produktion/Production & layout: VINNOVAs Kommunikationsavdelning/Communication Division Tryck/Print: EO Print AB, www.eo.se April 2004 Försäljning/Sold by: Fritzes Offentliga Publikationer, www.fritzes.se

VINNOVA är en statlig myndighet med uppgift att främja hållbar tillväxt genom utveckling av effektiva innovationssystem och finansiering av behovsmotiverad forskning. VERKET FÖR INNOVATIONSSYSTEM SWEDISH AGENCY FOR INNOVATION SY STEMS VINNOVA, SE-101 58 Stockholm Besök/Office: Mäster Samuelsgatan 56 Tel: +46 (0)8 473 3000 Fax: +46 (0)8 473 3005 VINNOVA@ VINNOVA.se www.vinnova.se