2019 IEEE Technology & Engineering Management Conference (TEMSCON)
Responsible Research and Innovation in Engineering
and Technology Management: Concept, Metrics
and Assessment
Lukasz Nazarko
Faculty of Business Management
Vilnius Gediminas Technical University
Vilnius, Lithuania
l.nazarko@ieee.org
ORCID 0000-0002-3596-3670
Abstract—Responsible Research and Innovation (RRI)
is a recent conceptual response of the European Union
to the challenge of aligning research and innovation (R&I)
activities with societal goals and purposes. This paper critically
analyzes this concept with a view of its coherence and applicability
to engineering and technology management. Authors discuss
the RRI relation to Technology Assessment (TA) and Corporate
Social Responsibility and review the possible approaches
to the measurement and assessment of R&I activities
in the context of RRI. Theoretical and practical obstacles to RRI
implementation are presented. Authors call for more groundwork
in clarifying the theoretical underpinnings of RRI. Furthermore,
they conclude that innovative business and industry may benefit
from self-reflection and self-assessment in the framework of RRI,
however there is still a need to refine available tools and develop
more industry specific approaches to responsibility in R&I.
Keywords — Responsible Research and Innovation,
responsibility, innovation, engineering, technology management,
Technology Assessment
I. INTRODUCTION
Responsible Research and Innovation (RRI) is a term that
has been popularized in recent years as a new integrated
approach to research and innovation activities with a view
to their ethical acceptability and social desirability.
It is promoted by the European Commission (EC) and has
a status of a cross-cutting priority in the European Union’s
Horizon 2020 Framework Program for Research
and Innovation. European Commission has by now supported
RRI-related projects with nearly €100 million (with project
participants from almost 50 European and non-European
countries).
The popularity of RRI in European science, technology
and innovation policy circles goes in pair with a considerable
scientific interest. Since 2009, when it was used for the first time
by Robinson [1], the term “responsible research and innovation”
has featured as a topic in more than 300 papers indexed
in Clarivate Web of Science.
The aim of this paper are following: 1) to critically analyze
RRI concept from the perspective of its relevance
This project has received funding from European Social Fund (project
No 09.3.3-LMT-K-712-02-0155) under grant agreement with the Research
Council of Lithuania (LMTLT).
978-1-7281-1139-1/19/$31.00 ©2019 IEEE
Borisas Melnikas
Faculty of Business Management
Vilnius Gediminas Technical University
Vilnius, Lithuania
borisas.melnikas@vgtu.lt
and applicability to engineering and technology management,
2) to relate the concept of RRI to more established frameworks
of Technology Assessment and Corporate Social Responsibility
(CSR), 3) to review metrics and indicators that could be used
to measure the degree of organization’s alignment with
RRI principles, 4) to discuss the possibility of developing
RRI assessment tools for organizations performing R&I
activities. Authors employ critical literature review, document
analysis and logical construction as methods of their research.
The work starts with the reflection on the meaning
of responsibility in engineering. Next, it presents the RRI
concept and outlines its novel elements as well unresolved
questions related to the understanding of RRI. Subsequent
sections of the paper describe the results of authors’ survey
of RRI-related metrics and indicators. They are evaluated
against their suitability for assessing R&I activities in industry.
II. RESPONSIBILITY IN ENGINEERING
AND TECHNOLOGY MANAGEMENT
Mitcham [2] reaches to the origins of engineering profession
and recalls that the term "engineer" dates to 14th century.
It originally referred to a designer and constructor of machines
for military use. Hence, engineering was originally military
engineering and engineers were subject to a hierarchy with the
unconditional duty of obedience. At that time, the engineer's
responsibility was defined by the execution of superior's orders.
Several centuries later, civil engineering emerged as a mature
technical discipline that specialized in the design
and construction of road, bridges, water and sanitation systems
as well as many other non-military infrastructures. This
development, however, did not change the subordinate nature
of engineers’ responsibility. Only at the turn of the 19th and 20th
century, when the Western social order moved from hierarchy
and obedience towards equality and self-interest,
did the engineering profession start to realize and assume
responsibility understood as a personal duty to care for public
health, safety and well-being [3]. Engineering associations
started formulating codes of conduct similar to those present
in medicine or law. The emergence of IEEE could also be seen
as an important event of that historical process.
 2019 IEEE Technology & Engineering Management Conference (TEMSCON)
Collins [4] claimed in 1973 that “the (…) view that engineers
as a group should blindly practice engineering without
professional concern over the critical social issues involved
in their work [was], hopefully, in the process of being
discarded”. He proposed to distinguish three dimensions
of engineer’s responsibility:
• individual responsibility – related to the possible
consequences of engineer's own work,
• group responsibility – related to the role
of technical societies and other forms
of associations,
• responsibility in public debate – to bring special
competence to the discussion on the issues arising
from the use of technology.
Such definition of engineering didn’t include deliberate
actions oriented at identifying societal needs and concerns that
await to addressed with scientific and technological means.
What makes the current innovation ecosystems different
from the past ones is their growing complexity and dynamism.
In consequence, their effect on society, economy, environment,
culture and values is increasingly difficult to predict. That calls
for a different (reframed) understanding of responsibility which
moves from “responsibility in and to society” towards
“responsibility for and with society” [5]. That shift underscores
the growing interdependence of stakeholders in R&I processes.
It requires defining socially desirable goals and purposes
of R&I (instead of just stating what it should not do). It also
entails broader and more intensive use of institutionalized
reflection, deliberation and anticipation instruments.
III. RESPONSIBLE RESEARCH AND INNOVATION –
THE CONCEPT
RRI is often defined as a “transparent, interactive process
by which societal actors and innovators become mutually
responsive with a view to the (ethical) acceptability,
sustainability and societal desirability of the innovation process
and its marketable products (in order to allow a proper
embedding of scientific and technological advances in our
society)” [6]. It is the ambition of RRI promoters to make the
concept relevant to various groups of actors engaged directly
or indirectly in research and innovation (R&I): researchers,
policy makers, educators, business and industry innovators
and civil society organizations. According to this and other [5]
[7] [8] definitions responsible research and innovation should
be:
• transparent,
• interactive and inclusive of all relevant
stakeholders,
• oriented at the exploration of future impact
(intended and unintended),
• ethically acceptable,
• socially desirable.
EC aims at fostering RRI by action in the following spheres
(policy agendas): 1) Ethics, 2) Gender Equality, 3) Governance,
4) Open Access, 5) Public Engagement, 6) Science Education.
That action should, according to Owen et al. [5], meet
the following process criteria: 1) Diversity and Inclusiveness,
2) Openness and Transparency, 3) Anticipation and Reflexivity,
4) Responsiveness an Adaptation (Figure 1).
Fig. 1. Resposible Research and Innovation – stakeholders, policy agendas
and process cirteria. Source: [9]
RRI is addressed at both (public and private) research
institutions and innovating businesses. The declared benefits
(supported by case studies) of RRI adoption in industry
are following: 1) enhancing corporate reputation, 2) lowering
risks of unintended consequences, 3) strengthening public trust
in the safety of products in the market. 4) increasing
competitiveness beyond short-term, 5) strengthening links with
customers and end users, 6) increasing acceptability of products
[10].
The above benefits can be achieved by: 1) recognizing
business’s responsibilities beyond profit alone, 2) reflecting
on values and assumptions underlying innovation processes,
3) identifying and addressing potential tensions between market
success and societally responsible product and service
development, 4) understanding the importance of inclusiveness
and diversity of innovation teams, 5) mapping out stakeholders
that need to be involved in the innovation process, 6) mapping
out issues that would benefit from RRI, 7) identifying what
practices need to change as a response to RRI paradigm [11].
There is a number of unresolved questions in relation to RRI.
They are discussed among others by Timmermans & Blok [12]
and Blok & Lemmens [13]. In addition, there is no clarity
as to the “division of labor” in RRI. On one hand, RRI concept
enumerates groups that are supposed to assume responsivity
for/in R&I activities. On the other hand, RRI seems to question
the notion that responsibility in the current world may be strictly
defined and assigned to groups and individuals according
to their social role. Since all "exercise the powers
of technoscience through their support for modern scientific
education and research or the utilization of and dependence
on technological products, processes and systems" [2], all have
become in some sense technology managers and engineers and
thereby should assume responsibility for the common good.
All stakeholders are expected to communicate the priorities,
values, principles, and norms that are a basis for their actions
and contribution to the scientific-technological processes [14].
Such assertion gives way to promising interpretations of RRI
 2019 IEEE Technology & Engineering Management Conference (TEMSCON)

as meta-responsibility [15] [16] and networked responsibility of a product to the market, whereas CSR concerns the full
[17]. product life cycle [19].
Apart from theoretical gaps and ambiguities, another major Technology Assessment, in its classical form, “address those
problem with the RRI concept consists in putting it in practice. technology aspects which are of public interest and therefore
It is not at all clear how this – sometimes rather sophisticated – should be made subject to political reasoning and democratic
discourse should be translated into concrete actions and attitudes decision making” [20]. It could be labelled as public TA
in day-to-day operations of innovators, engineers and is typically carried out by specialized institutions. However,
and technology managers. Inzelt & Csonka [18] have compiled despite the inherent tension between the imperative
a set of barriers to the implementation of RRI in business of transparency and inclusion on one hand, and the desire
practice (Table I) to protect private knowledge for the sake of competitive
advantage on the other, there appears a space for TA in business
TABLE I. BARRIERS TO RRI IMPLEMENTATION IN BUSINESS PRACTICE sector (private TA) [21]. This has to do with the increasing
importance of sustainability reporting and CSR through which
Barrier group Details companies try to position themselves as responsible actors
in the society. In Figure 2 a simple conceptual model
• negative attitude towards the idea of RRI
of connecting RRI and technology management is presented
• low level of understanding of the RRI concept
• no culture/structure of knowledge sharing
with TA as a binding element.
in the organization
Capabilities- • no willingness to commit time or other
related barriers resources for RRI processes
• half-hearted shift towards open innovation
• lack of competencies in dealing with RRI
(no tradition and expertise in engaging
stakeholders)
• management models that leave little/no space
for RRI (heavily growth- or sales-oriented)
Other business
• unwillingness to sacrifice competitiveness
priorities
(e.g. low price) for the sake of RRI
• fear of risk connected to innovation
Cross-country
• applicable especially to global companies
law, regulatory,
of non-European origin that lack the European
and cultural
governance system of RRI
differences
• non-existence of applicable training resources
Lack of
and units that may equip companies with
education
relevant RRI competencies
• focus on short-term performance instead
Short-termism of long term goals whereas RRI lies
in the sphere of "long term"
Intellectual • intellectual property strategy not in line with
property regimes open access model

IV. RRI IN RELATION TO TECHNOLOGY ASSESSMENT
AND CORPORATE SOCIAL RESPONSIBILITY
One cannot avoid relating RRI to other better-established
concepts that deal with responsibility in engineering and
technology management, namely Corporate Social
Responsibility (CSR) and Technology Assessment (TA).
When it comes to the relation of RRI and CSR, one can make
the comparison on two levels. The first level concerns
the relation to social needs, concerns and values. On that level
RRI may seem to be a further reaching concept because
it focuses on societal desirability, not only on avoiding harmful
impact (as it is the case in traditional CSR activities). However,
contemporary CSR portfolios of many companies go beyond
"doing no harm" towards the creation of positive social impact.
The second level of comparison is related to the relevance
of a particular concept to the consecutive phases of product life
cycle. In this context RRI may be considered narrower in scope
since it concentrates on all phases prior to the introduction
Fig. 2. RRI, Technology Management and Technology Assessment –
conceptual connections (Source: [22])
It is evident that RRI has not been born into the void but
rather into a space occupied by many concepts, frameworks,
paradigms and policy agendas related to research, innovation,
ethics and society. There is no agreement as to how much
novelty the RRI concept brings. Some suggest that “RRI offers
a substantive, complementary and immanent reference that other
paradigms did not offer” [23]. Others see it rather
as an umbrella concept (meta-responsibility) integrating CSR,
engineering ethics and Technology Assessment.
V. RRI METRICS AND ASSESSMENT
Traditional management principle stating that you cannot
manage what you cannot measure calls for metrics and scales
that would make the elusive concept of R&I responsibility
(more) manageable. In the market setting, all types
of innovation (product, process, marketing, organizational
 2019 IEEE Technology & Engineering Management Conference (TEMSCON)
innovations) may be a subject of assessment in the RRI
framework.
Literature review and the analysis of documents and reports
produced in the framework of various EU-sponsored projects
lead to the conclusion that the available RRI-related metrics
and indicators have little applicability to the innovation process
in industry (except gender related indicators). They
are definitely oriented at research institutions, research funding
agencies and institutions involved in fostering the public
understanding of science [24].
The review performed by the authors resulted, however,
in the identification RRI self-assessment and self-reflection
tools oriented at business actors. Karim project offers
responsible innovation criteria together with "responsible
innovation flash diagnostic" [25]. RRI Tools project offers
a very comprehensive self-reflection tool organized around RRI
policy agendas. It is available both on-line and off-line [26].
Responsibility Navigator developed in the framework
of ResAgorA project presents ten principles organized into three
RRI-related dimensions: 1) Ensuring Quality of Interaction,
2) Positioning and Orchestration, 3) Developing Supportive
Environments. The ORBIT Self -Assessment Tool, in turn,
is tailored to the needs of the UK ICT community[27].
Stahl et al. [28] propose a five-level RRI Maturity Model
with the following stages: 1) Unaware, 2) Exploratory/reactive,
3) Defined, 4) Proactive, 5) Strategic. Worth mentioning,
especially for the North American readers, is STIR (Socio-
Technical Integration Research) project which
is an experimental platform for scientists and engineers
to incorporate the RRI thinking into their activities.
VI. CONCLUSIONS
Jonas [29] considers responsibility to be a function
of knowledge and power. As our knowledge about
and the power over matter and space (thus the power to affect
people on the whole planet, the future generations,
and the entire nature [30]) are increasing so is our
responsibility. Paraphrasing Collins’ statement [4] it may
be conceded that the narrowness in technical/scientific
education and a high degree of specialization handicaps many
innovators in evaluating the relationship of their work
to broader societal and environmental questions. What was true
in the 1970s is even more evident in the second decade
of the 21st century. The exponential growth in complexity
of socio-technical systems and the inability to predict all
interactions therein spur repeated efforts to establish paradigms
and frameworks that are supposed to steer innovation towards
solving the grand civilizational challenges and avoiding
negative consequences of scientific and technological progress.
RRI should be seen as the most recent attempt to tackle that
long known problem.
This paper offers an overview of the RRI concept
and assesses its applicability to engineering profession
and technology management. The conducted research resulted
in the conclusion that more theoretical/conceptual groundwork
is still needed in clarifying the ontological and epistemological
underpinnings of RRI. Although Gianni suggests that RRI
is by nature an under-construction project and it should be seen
as a “performative framework that can never be accomplished”
it is authors’ conviction that the ambiguities and contradictions
currently present in the RRI discourse should be dealt with.
On the conceptual level, further research may include the study
of links between Future-Oriented Technology Analysis (FTA)
and RRI [31]. In the application field, the development of RRI
benchmarking tools with the use of multi-criteria decision
making [32] and approaches traditionally used in productivity
analysis [33] [34].
At the current state of the development of RRI concept,
it seems more fitting (in terms of used language
and operationalization) to public research institutions, research
funding agencies, universities, etc. Its applicability
to innovative business and industry consists mainly
in the opportunity for self-reflection and the possibility
to perform self-assessment. That creates potential
for benchmarking and the creation of industry specific RRI
scorecards. Nevertheless, a big research gap still exists in field
of understanding responsibility in business and in developing
relevant RRI metrics and indicators.
REFERENCES
[1] D.K.R. Robinson, “Co-evolutionary scenarios: An application
to prospecting futures of the responsible development
of nanotechnology“, Technological Forecasting and Social Change 76(9),
2009, pp. 1222–1239.
[2] C. Mitcham, “Technology and the Burden of Responsibility“ in Values
and Ethics for the 21st Century, BBVA, 2012, pp. 141-164.
[3] R. McKeon, “The Development and the Significance of the Concept
of Responsibility”, Revue Internationale de Philosophie 11(1), 1957,
pp. 3–32.
[4] F. Collins, “The Special Responsibility of Engineers”, Annals of the New
York Academy of Sciences 196 (10), 1973, pp. 448–450.
[5] R. Owen, J. Stilgoe, P. Macnaghten, M. Gorman, E. Fisher, D. Guston,
"A framework for re-sponsible innovation", in Responsible innovation,
R. Owen, J. Bessant, M. Heintz, Eds. London: John Wiley & Sons, 2013.
[6] R. Von Schomberg, “Prospects for technology assessment in a framework
of responsible research and innovation” in Technikfolgen abschätzen
lehren, M. Dusseldorp, R. Beecroft, Eds. Wiesbaden: VS Verlag
für Sozialwissenschaften, 2012, pp. 39-61.
[7] R. von Schomber, “A vision of responsible research and innovation“,
in Responsible innovation, R. Owen, J. Bessant, M. Heintz, Eds. London:
John Wiley & Sons, 2013.
[8] J. Stilgoe; R. Owen, P. Macnaghten, “Developing a framework
for responsible innovation”, Research Policy 42(9), 2013, pp. 1568–1580.
[9] RRI Tools project, https://www.rri-tools.eu/ (Accessed 31 March 2019)
[10] Benefits of Responsible Research and Innovation in ICT for an ageing
society, 2017, http://www.responsible-industry.eu (Accessed 31 March
2019)
[11] Learning Outcomes, RRI Tools Project, https://www.rri-
tools.eu/documents/10184/193151/RRIToolsTraining-
LearningOutcomes.pdf/ (Accessed 31 March 2019)
[12] J. Timmermans, V. Blok, “A critical hermeneutic reflection
on the paradigm-levelassumptions underlying responsible innovation”,
Synthese, 2018, pp. 1–32.
[13] V. Blok, P. Lemmens, “The emerging concept of responsible innovation.
Three reasons why it is questionable and calls for a radical transformation
of the concept of innovation”, in Responsible innovation Vol. 2,
B-J. Koops, I. Oosterlaken, H. Romijn, T. Swierstra, J. van den Hoven,
Cham: Springer, 2015, pp. 19–35.
[14] E. Schuurman, Technology and the Ethics of Responsibility,
https://www.metanexus.net/technology-and-ethics-responsibility/
(accessed 31 March 2019)
 2019 IEEE Technology & Engineering Management Conference (TEMSCON)
[15] BC. Stahl, “Responsible research and innovation: The role of privacy
in an emerging framework”, Science and Public Policy 40, 2013, pp. 708–
716.
[16] BC. Stahl, J. Timmermans, S. Rainey and M. Shaw, “Ethics in innovation
management as meta-responsibility” in The Routledge Companion
to Innovation Management, J. Chen, A. Brem, E. Viardot, PK. Wong,
Routledge, 2019.
[17] J. Ceicyte, M. Petraite, “Networked Responsibility Approach
for Responsible Innovation: Perspective of the Firm”, Sustainability
10(6), 2018, 1720.
[18] A. Inzelt, L. Csonka, “The Approach of the Business Sector
to Responsible Research and Innovation (RRI)”, Foresight and STI
Governance 11(4), 2017, pp. 63–73.
[19] L. Nazarko, “Responsible Research and Innovation in Industry: “From
Ethical Acceptability to Social Desirability”, in Corporate Social
Responsibility in the Manufacturing and Services Sectors, P. Golinska-
Dawson, M. Spychała, Eds., Berlin: Springer, 2019, pp. 127-138.
[20] A. Grunwald, “Technology Assessment in Practice and Theory”, Oxon-
New York: Routledge, 2019.
[21] L. Nazarko, "Technology assessment in construction sector as a strategy
towards sustainability", Procedia Engineering 122, 2015, pp. 290–295.
[22] L. Nazarko, “Future-Oriented Technology Assessment“, Procedia
Engineering 182, 2017, pp. 504–509.
[23] R. Gianni, “Responsibility and Freedom: The Ethical Realm of RRI“,
London: ISTE, 2016.
[24] MoRRI - Monitoring the Evolution and Benefits of Responsible Research
and Innovation, Report, https://morri.netlify.com/ (Accessed 31 March
2019)
[25] Introduction to a guide to entrepreneurs and innovationsupport
organization, Karim project, http://www.nweurope.eu/media/1118
/guide_online.pdf (Accessed 31 March 2019)
[26] Self-Reflection Tool, RRI Tools, https://www.rri-tools.eu/self-reflection-
tool (Accessed 31 March 2019)
[27] B. Stahl, The ORBIT Self-Assessment Tool, ORBIT Journal 1(2), 2017.
[28] BC. Stahl, M. Obach, E. Yaghmaei, V. Ikonen, K. Chatfield, A. Brem,
“The Responsible Research and Innovation (RRI) Maturity Model:
Linking Theory and Practice”, Sustainability 9, 2017, 1036.
[29] H. Jonas, “The Imperative of Responsibility: In Search of an Ethics
for the Technological Age“, Chicago: University of Chicago Press, 1984.
[30] H. Jonas, “Technology and Responsibility: Reflections on the New Tasks
of Ethics”, Social Research: An International Quarterly, 40 (1), 1973,
pp. 31–54.
[31] K. Halicka, “Innovative classification of methods of the Future-oriented
Technology Analysis”, Technological and Economic Development
of Economy 22(4), 2016, pp. 574-597.
[32] A. Mardani, A. Jusoh, K. Halicka, J. Ejdys, A. Magruk, U.N.U. Ahmad,
“Determining the utility in management by using multi-criteria decision
support tools: a review”, Economic Research-Ekonomska Istraživanja
31(1), 2018, pp. 1666-1716.
[33] E. Chodakowska, “Construction of the Environmental Performance Index
using DEA”, Quantitative Methods in Economics 15, 2014, pp. 296-306.
[34] E. Chodakowska, J. Nazarko, “Environmental DEA method for assessing
productivity of European countries”, Technological and Economic
Development of Economy 23(4), 2017, pp. 589-607.