USING RESEARCH METHODS IN HUMAN COMPUTER INTERACTION TO DESIGN TECHNOLOGY FOR RESILIENCE

ABSTRACT

Research in human computer interaction (HCI) covers both technological and human behavioural concerns. As a consequence, the contributions made in HCI research tend to be aware to either engineering or the social sciences. In HCI the purpose of practical research contributions is to reveal unknown insights about human behaviour and its relationship to technology. Practical research methods normally used in HCI include formal experiments, field experiments, field studies, interviews, focus groups, surveys, usability tests, case studies, diary studies, ethnography, contextual inquiry, experience sampling, and automated data collection. In this paper, we report on our experience using the evaluation methods focus groups, surveys and interviews and how we adopted these methods to develop artefacts: either interface's design or information and technological systems. Four projects are examples of the different methods application to gather information about user's wants, habits, practices, concerns and preferences. The goal was to build an understanding of the attitudes and satisfaction of the people who might interact with a technological artefact or information system. Conversely, we intended to design for information systems and technological applications, to promote resilience in organisations (a set of routines that allow to recover from obstacles) and user's experiences. Organisations can here also be viewed within a system approach, which means that the system perturbations even failures could be characterized and improved. The term resilience has been applied to everything from the real estate, to the economy, sports, events, business, psychology, and more. In this study, we highlight that resilience is also made up of a number of different skills and abilities (self-awareness, creating meaning from other experiences, self-efficacy, optimism, and building strong relationships) that are a few foundational ingredients, which people should use along with the process of enhancing an organisation's resilience. Resilience enhances knowledge of resources available to people confronting existing problems.

Keywords:
Mixed Methods Research; Human-computer Interaction; Resilience; Resilience Strategies; Technology; Artefacts; Information Systems

1. INTRODUCTION

Human-Computer Interaction is a multidisciplinary arena which draws on the fields of computer science, psychology, cognitive science, and organisational and social sciences in order to understand how people use and experience interactive technology. This multidisciplinary field makes use of qualitative and quantitative research methods either to gather or to analyse information to be used. We are convinced that this is a trivial matter as research methods for HCI. However, the methods used were necessary to have guidance about what steps we needed to take to optimize organisations' resilience. Conversely, they were important to align technology capabilities with organisations' requirements.

In the HCI community and in the design practice community, the term design research is generally used to refer to what the upfront research practitioners do in order to ground, inform, and inspire their product development process. Design research has many methods, drawn from other disciplines or created by designers over the years. These methods can be roughly divided into three categories: observations, interviews, and activities, including having subjects make things and self-report on their activities (Shaffer 2009). We used design research to uncover people's behaviours, motivations, and goals in order to design for them. Understanding of peoples' approach and ways of thinking is significantly essential while we are designing for them. The relationships between people and technologies and environment, in which they participate, were subject to this research in order to determine how they understand organisational environment to improve resilience strategies.

Current research in resilience has focused on two areas, organisational and information technology. Organisational resilience considers the ability of an organisation to survive in the face of threats, including the prevention or mitigation of unsafe, hazardous or detrimental conditions that threaten its very existence (Hollnagel, 2006). Information technology resilience considers stability and quality of service in the face of threats to the computing and networking infrastructure. In considering the areas of resilience, we will be interested in resilient design, which can be used to cover user's performance within an organisation.

2. THE CONTEXT

In this section we present some definitions of resilience found in literature necessary for organisations to face unexpected situations; we highlight the importance of defining strategies and planning of the issues and events from an organisation in order to reach the best solution. Then, we give an overview of the research methods we used to collect and analyse the data from each project we did.

2.1 Resilience Strategies

Resiliency is a concept, rooted in psychotherapy and social psychology (Hind et al. 1996, Ruttner et al. 1990) material science (Sheffi 2005) and ecology (Holling et al. 2002) and it is fundamentally about the "robustness" of systems (Beinhocker 1999), (Deevy 1995). The sense of valued identity, common purpose and shared beliefs (Coutu 2002), (Freeman et al. 2004), (Hirschhhorn et al. 1992) are central to resilience. Resilience has been defined as the ability to cope with an unexpected situation, to "bounce back" (Sheffi 2005, Hirschhorn et al. 1992). In other words, organisation resilience is the ability to rapidly adapt and respond to business disruptions and to maintain continuous business operations, be a more trusted partner, and enable growth. Discussions of resilience have centred on properties related to human action in responding to unexpected events: redundancy, resourcefulness, communication ability, self-organisation, improvisation, role switching, and information seeking (Sheffi 2005, Hirschhorn et al. 1992). Resilience is also associated with creativity and innovation.

Hollnagel (2010) refers resilience as competency more than an organisational resource, which is seconded by Lengnick-Hall et al. (2011), i.e., set of routines that allow recovering from adversity and obstacles, and to create new opportunities and go beyond the point of initial equilibrium. According to Bhamra et al. (2011), this organisational competency depends on the availability of resources. Therefore, different organisations (or the same organisation during its lifetime) present different degrees of resilience according to the resources they possess to adapt their activities and avoid the variability of its environment.

Resilience strategies start with understanding exactly what the organisation needs in order to survive unexpected events and plan ahead for challenges that could come at any time. Resilience strategies should be planned to face possible disrupted situations, for example: several implementing alternative resources or solutions for the use of these technologies: analysis of the usability of the adopted technologies; new technologies acquisition to eliminate the unnecessary ones. Resilience planning should also consider the human factor: understand how people deal with the technology resources and how they respond in difficult situations; sometimes people make mistakes due to missing training, the absence of alignment between people skills and technology use is a frequent reality within organisations; paying attention that some errors are in technology design, which should be overcome by considering user centred design approaches.

Resilience is an on-going process that requires people in taking a number of steps. The strategies for increasing resilience are: having a supportive relationship within and outside the people in the organisation (these relationships will build trust among the intervenient in the process); the capacity to make realistic plans and take steps to carry them out; confidence in our strengths and abilities; skills in communication and problem solving are some of the ingredients to improve resilience. We use tiers, which are an architectural approach, for example, the general requirements that could be applied to the development or use of any new IT system. The alignment of those requirements with the technologies in use and user's performance and the provision of technical characteristics, usability components and human resources skills will give us resilience levels to be added to the planning process and help to maximise our investments and efforts. Defining, developing, and maintaining resilience tiers and associated resilient capabilities have a number of benefits, for example: Better mission-to-technology alignment; Clear rationalisation of investments in resilient capabilities; Assistance in gap analysis across mission, service, and technology domains (...) (Denis, 2015).

2.2 Research Methods in Human Computer-Interaction

The focus of human computer-interaction (HCI) is the investigation of relationships between computer technology, human activity and society. We are interested in the development of artefacts, services and systems that improve people's lives, in general, and we specifically researched how people worked, played, and communicated within groups, organisations, and social structures. Then we designed, created, and evaluated technologies and tools to support humans and their social activities. There is a range of research methods used in HCI. As this field is multidisciplinary, we need all the research methods used in almost all social sciences, along with some engineering and medical research methods. Various methods and tools are being used within organisations to improve the understanding of user task requirements to support design process and evaluation as well as because each project has different characteristics and objectives. User-centred design is a multidisciplinary design approach based on the active participation of users to improve the understanding of user and task requirements, and the iteration of design and evaluation (Mao et al., 2005; Eklund et al., 2008). The most used user centred design methods are: field study, user requirements, iterative design, usability evaluation, task analysis, focus groups, formal heuristics evaluation, user interviews, prototype without user testing, surveys, informal expert review, card sorting, participatory design (Schindlholzer 2014).

There are some techniques, described by design research, which explain how to add context and insight to design process. These techniques are known as user research. Design research included the careful analysis of findings, giving them a direction, and looking for patterns (Laurel, 2003). The tools of design research are both quantitative and qualitative methods. However, most design research is qualitative, not quantitative.

This paper presents the use of mixed methods research applied to the human computer interaction field. Mixed methods are generally used when we consider the use of both quantitative and qualitative techniques (Creswell 2014). According to Johnson (2004) mixed research deals with compatibility and pragmatism. The idea is that quantitative and qualitative methods are compatible and pragmatism meaning "researchers should use the approach or mixture of approaches that works the best in a real world situation". Qualitative research is more subjective, based on smaller, targeted sample sizes, and is concerned more with how and why questions. (Saffer, 2010) Quantitative research, on the other hand, is often about large, random, statistically significant sample sizes and is designed to answer questions. Quantitative methods permitted to obtain numerical data concerning, for example and in general, the number of prospective users for our applications, the number of those that were used in similar applications, the tendency of use, etc. Both qualitative and quantitative research, in combination, provided a better understanding of the research problem. Although mixed methods research is generally concerned with the combination of quantitative and qualitative methods, we followed the paradigm of mixing almost qualitative methods. According to Brannen (2005), the methods may be a mix of qualitative methods. We used focus groups, surveys and interviews for conducting user research, since they are suitable for answering questions about what, why or how to fix a problem. These methods permitted categorising attitudes and providing a view of what people think about the interface/application we will propose. Our cases report, mainly, focus groups, questionnaires and interviews as methods to collect data to enrich different interface and systems design and development. Conversely, various methods and tools are being used within organisations to improve the understanding of user task requirements to support the design process and evaluation as well as because each project has different characteristics and objectives. Each project environment must be considered which is in accordance with the numerous concepts that emerge from definitions of organisational resilience including knowledge of the environment, level of preparation, anticipation of perturbations, capacity to deploy resources, degrees of adaptation, capacity of recovery, etc. (McManus et al, 2008).

Figure 1 provides an overview of user-centred design techniques (Young 2008). The research activities are in darker colour. The design team is responsible for the analysis of user research. The projecting analysis techniques include the design of personas, mental models, storyboards, etc.

Figure 1
User centred design techniques

Generally, designers use these research methods to obtain information about subjects and their environment. Otherwise, they might not know them, firstly, because of their background or experiences or due to the specificities of the user and/or environment. Both methods were important within our study. The focus on qualitative methods of research gave us the opportunity to get data related to user's motivations, expectations, and behaviours, which are extremely valuable to interaction designers. We asked questions, took notes and learned everything we could about the user or client during the projects' design. Norman (2013) argued: "we tend to project our own rationalisations and beliefs onto the actions and beliefs of others". Figure (2) represents a starting point framework, which we defined to guide our research and to figure out the data collection steps we should follow. Implicitly, resilience strategies are included, namely: implementation of alternative resources or solution; analysis of the adopted technologies' usability; new technologies acquisition; considerations about human factors (how people use technologies, how they respond in different situations).

Figure 2
Initial framework for data collection

Focus group was the method to explore opinions about a specific product and topics, e.g., in our application development for musicians and in our project about the centre for the elderly. In the field of human computer interaction, they are used to explore user perspectives in systems and their usability. Focus group tends to be less useful for usability purposes, for a variety of reasons, but it provides a top-of-mind view of what people think about a brand or product concept in a group setting. It was a prevailing tool for the systems development; however, it was not the only source of information to get data about user behaviour and to discover what they wanted from the systems.

Surveys represent one of the most common types of quantitative research. Survey sampling is particularly useful when the population of interest is very large or dispersed across a large geographic area. Survey research is widely used in human-computer interaction (HCI) to measure users' attitudes and collect product feedback. We did online surveys to gather feedback about a learning tool proposal and to measure user's satisfaction about framework's use. In survey research, the researcher selects a sample of respondents from a population and directs a standardised questionnaire to them. Questionnaires and surveys are complementary tools: A questionnaire is a research tool that uses questions in the gathering of information from different respondents while a survey is the systematic collection of information from different individuals. Also, a questionnaire is a survey tool while a survey is the process of using questionnaires to gather information. A survey is broad, while a questionnaire is a specific type of gathering of information. The questionnaire, or survey, can be a written document that is completed by the person being surveyed, an online questionnaire, a face-to-face interview, or a telephone interview. In this study, the goals of the conducted surveys were the measurement and categorisation of attitudes or the collection of self-reported data that could help track or discover important issues to address the application or interface development.

Interviews with interviewees' consent, by asking questions that explore a wide range of concerns about a problem, to give them the freedom to provide detailed responses. Interviews were used in almost any phase of the project, from initial exploration to requirements gathering, evaluation of prototypes, and summative evaluation of completed interfaces/systems. Interviews were applied to all the projects. In the case of project 3, interviews followed the survey to complement the gathered data. When the interviews were concluded a new phase started, as well data transcription and analysis. From that moment on, different codes were defined to give rise to several categories. This analysis process was made with other spreadsheet and from there charts and diagrams were constructed. For project 4 we prepared additional questions, to those included in the survey, regarding musical learning instrument. Our goal was to get more information about the interviewees to understand user needs and desires in our proposed learning application.

3. THE ROLE OF TECHNOLOGY IN ENHANCING RESILIENCE

Every day technologies are used to equalise opportunities in the connected world. Today's globalised world has been largely made possible by innovations in technology -the mobile phone and the Internet have changed people's behaviours. The very same technologies, if used innovatively and with imagination, can provide resilient and affordable capabilities to more effectively match emerging needs. Online maps, location-based services, mobile apps and social media, general or customised applications, in particular, can all help bring people's attention and behaviour back towards a collaborative contribution and commitment. Technology can help drive awareness and engagement, and promote convenience, in all of these activities.

From an Information Technologies perspective, organisations face complex challenges when dealing with technologies. The role of resources, and especially IT resources has not received much attention in acts of resilience. However, there is a huge potential in using technology to stimulate behaviours, which can lead to economic development.

We found some examples of the use of technology to strengthen resilience. Abi Weaver of the American Red Cross (2015) suggests that emerging technology solutions can strengthen people's abilities to cope with disasters. Rieger et al. (2009) presents a control system application to provide a basis for resilient control system research. Wright et al. defined a set of recommendations as guidance for designing and maintaining enterprise resilience concerning cyber security operations - cyber crime. Boyes (2013) argued about the resilience in intelligent or smart buildings. They especially examine different sources of threat across a building life cycle. Lal et al. (2006) focus on web technologies and connectivity as tools for an organisation to implement a framework, which leads to organisational resilience. Thus, technology is a tool for capturing, organising, and diffusing organisational knowledge. Conversely, technology can be the solution to implement new ways of dealing with different issues or events and even to sort some information systems gaps and technology needs.

Technology use and resilience were the approach we wanted to deal with in the presented case studies (projects). We made two main questions, which we wanted to see, answered at some phase of the study: how people used technology to restore collaborations when there was a major environmental disruption. And, how technology played a major role in providing people with alternative resources to reconstruct, modify, and develop new routines, or patterns of action, for work and socialising.

4. THE STUDY

This study provides some design guidelines for building and applying interventions to increase resiliency. Many practices that promote resilience already existed in some of the organisation, but they needed to be identified, improved, where necessary, and then, aligned within an overall capacity-building strategy. We focus on IT tools to improve services and the ability to respond to any threat or opportunity. Hollnagel et al. (2010) argued that resilience can be focused on four abilities: a) to respond to what happens, b) to monitor critical developments, c) to anticipate future threats and opportunities, and d) to learn from past experience - successes as well as failures. Working with the four abilities provides a structured way of analysing problems and issues, as well as of proposing practical solutions (concepts, tools, and methods).

The study comprises the description of four projects where we used focus groups (to get information approaching participants' experiences about the impact that changes could bring) mainly on (Project 1 - A Framework for e-government, Project 4 - Learning tool for musicians). We did surveys on (Project 2 - User centred healthcare design, Project 3 - The Framework Analysis in IT Companies, Project 4 - Learning Tool for Musicians), and interviews in all of them, to collect and analyse the data from the organisations to improve their levels of resilience.

The fields of the presented organisations are: two of them are organisations concerned with citizens' quality of life services, another one is specific to organisations that do software development and the last one is a system proposal to musicians. Furthermore, the four projects were in different fields, they are representative of the resilience strategies application to improve the organisations' performance.

The results of the data analysis, in some of the cases, ended with a proposal and development of a new technological artefact. For the whole projects we followed the user centred design approach. The goal was to design the development of an interface/system, which could be an act of resilience. We argue that for any interface design it is necessary to consider the user because we wanted to be sure that our development would fit user needs. For example, if the interface is designed for information systems users, it is important to recruit participants who are familiar with this subject; if the concern is with an interface for musicians users, it is important to recruit such users to the study. The target population help, for example, in HCI context, to the changes to be made on the interaction design, interface features and so on. Our design process focused on usability goals, user characteristics, environment, tasks, and workflow in the design of the artefact.

Project 1 - A Framework for e-government

The mission of services for the elderly is to enable our elders to live wtih dignity and to provide necessary support for them, to promote their sense of belonging, sense of security and sense of worthiness. The goals of the Portuguese social security system are to oversee the activities of non-public institutions for social solidarity and to improve the quality of operations and services as well as the interaction process of the collection, requirements and information system aligned with those of the social centres for the elderly. The main goal of this project was to understand how information systems of different social centres for the elderly are aggregated. The case study we describe is a private institution of social solidarity. Let's call it SCLF. The central objective of this centre is to help clients who do not succeed to have, in their homes, the support conditions for the necessary care for a good quality of life. The SCLF offers other different services: residential structure for aged people, home support services, day care centre, physiotherapeutic services and entertainment activities for children up to 10 years old. The centre is organised as presented in figure 3. Two main groups can be distinguished, both having in charge, the technical director and the services director. The technical director area is divided into the technical bureau and the offer that the nursing house gives, namely the leisure centre activities (LCA), day care centre (DCC), home support service (HSS) and the residential structures for the elderly (RSE). These services have different types of workers: direct help providers and service assistants; the technical bureau has the health office with a physiotherapist, a doctor and a nurse. The social office has a sociologist and in the leisure office there is a social educator. In the area supervised by the service director, we have stocks management activities and the secretariat whose tasks are developed by the director. The kitchen has six cooks and the laundry is where service assistants do ther tasks.

Figure 3
SCLF's Organogram

Project 2 - User Centred Healthcare Design

There is a considerable interest in health services' improvement to give the best service for society. Besides the opportunity for change and amelioration, it is not always easy to introduce adjustments, involving a large number of employees, patients, and other stakeholders. To circumvent this difficulty, we defined strategies and established partnerships with the staff management of a Portuguese Health Centre. They were open to collaborating with the academia helping them to identify and suggest some solutions to sort encountered problems. This was not an easy task, since work analysis in a health centre is not just to make a brief collection of data and return a functional program on what so ever. The main goal is to understand the real needs of doctors, nurses and other service providers and then, to provide a really useful, complete and desirable solution. It is also crucial to realize that there are several difficulties that most of the stakeholders have in relation to information technology. This means that the solution should be conceived as something intuitive and easy to use within the parameters collected. From the data collection, and beginning of its analysis, we encountered some problems that will be stated in section (5). The health centre of this case study is relatively new. Its facilities are new. The health centre serves about 160 patients daily and employs over 30 employees with different backgrounds and specialities. All general service centres operate on the basis of the interaction between the user and the reception staff. The patient enters the health centre and the first procedure that follows is to get a ticket. This ticket contains all the information regarding the service order and time. The reception consists of a row of six separate service counters. After getting a ticket, patients go the waiting room where there are interfaces that allow them to view services, queues, and the ticket counter. Thus, it is ensured all of them are served, as well as the expected time for each activity. Employees give the information, queries and other data from computers. To know their place of care, patients have to pay attention to the monitors in the waiting room, which give information through the corresponding service number and the counter. Employees made the call for specific queries. There is no indication for time delay.

Project 3 - Frameworks in Information Technologies Companies

In last few years, the business market has taken advantage of new technologies as a special factor for better productivity and to reach new customers. The increased demand for information technologies (IT) has put pressure on IT suppliers to provide products and services of greater complexity on a reduced time. IT companies have supported their customers' claims, and met their expectations, as well as, considered profits, and then a solution is found: for many companies one of the ways to be different and to respond to this challenge has been the development of Frameworks, which allow using the accumulated knowledge and experience that exist within a company. Conversely, it permits to advance faster with their products. According to (IBM, 1997) "A Framework is a set of blocks of pre-fabricated software that programmers can use to apply, or change specific computing solutions. (...) With the Frameworks the software developers do not need to start from beginning each time they start building an application. Frameworks are built from a collection of objects, so far as the design codes to be reused." In this research, we tried to analyse the impact of these types of tools inside companies, and we wanted to understand and identify: the segment markets where the Companies' Frameworks were used furthermost; the decision factors considered for the choice of a framework and the customers' satisfaction; the advantages of using Frameworks for enterprise businesses as well as for the development teams; the key requirements in the creation of Frameworks, and the areas of improvement on the Frameworks.

Project 4 - Learning Tool for Musicians

If we search on the Internet, there is a considerable amount of websites and information about interactive applications for learning music. However, there are some missing aspects in them, for example: there is a lack of direct interactivity with the player; a deficiency of a consistent method; absence of consideration about the player-developed skills throughout the learning process, etc. The main goal within this project was to develop a technological application for musicians that solve some of the encountered problems on other disposable systems. We wanted to understand the frequency of the instrument practice, the needs and motivations for its learning process and the importance given to learning from a teacher vs. learning alone as well as class-based learning vs. distance learning in this specific context. Conversely, we aimed for a user-centred application's development. Thus, a questionnaire was created to evaluate the musician community towards new ways of learning music and some semi-structured interviews were designed to both music professionals and amateurs. In addition, we did interviews whose focus was on learning practices, self-perception practices, relevant questions about technology use and relevant issues for the application development.

4.1 The Applied Methods

In project 1, focus group was a discussion of six participants guided by the researcher within this specific project. The researcher used a script created to initiate and structure the discussion. The approaches to gather information about situations, users, customers, collaborators and other stakeholders were institutional documents, attitudes from the community (nursing houses and institutions of social solidarity) and software applications in use. Then, we also did a questionnaire to verify if users agreed with the functionalities of our application. We intend to have suggestions about our layouts, functionalities, usability, and user's own comments. Around 20 semi-structured interviews were made to allow the interviewees to have some space to comment and/or present their concerns. The interviews were in a person-to-person interaction form. They were conducted in different social centres, private organisations, with different members. Firstly, we prepared the text for the interviews and we selected the staff members of each institution according a defined profile. After each set of interviews with people from the same profile (technical and/or services director) in each of the five institutions, we analysed the data and started to design the first sketches of the application. Then, we interviewed the information systems technician of each institution. Another phase of interviews with previous staff members was made to consolidate and correct the gathered information. Finally, we interviewed the administrative staff that would be in charge of the application use. Sixteen questions were designed to gather information from the interviewees, either to understand the procedures and interactions in and out the institutions or to know the type of complaints about work situations experienced. We were interested, namely, to identify the kind of information that was necessary for users' management; the compulsory information to be exchanged between the institution and the social security system; the access points to consult information; people that should be involved in the process and the output documents that should be created. In parallel with the previously collected information, we also wanted to understand the difficulties that staff, in nursing homes, had when dealing with the whole amount of information that is spread all over the institution.

Project 2

In this project a combination of research methods was used as a literature review, clinical interviews, observations, discussions with doctors and counsellors, and patient interviews. We did also a questionnaire to complement the collected data. The data was gathered from around 200 people. Notes were written from observations, and interviews were video recorded. The conversations were transcribed preserving the context in which things were said and done. It is planned to interview more people before the phase of theory generation either to validate the data or to complement it and arrive to data saturation. Grounded Theory Method was chosen since it has been used, frequently, in healthcare study domains and conversely, because it gives guidelines and grounding than most approaches. In Grounded Theory Method the data is collected, and a constant comparative method is analysed until saturation and theory emerged. The data was, firstly, analysed using Zachman Ontology because we consider this ontology well suitable to have a picture of an organisation. This Framework is frequently used for enterprise architecture. The Zachman Framework is composed of the intersection of two lines, the first consisting of the questions with primitive foundations of communication and the second derives from the transformation of abstract ideas in instances. This Framework is typically described as a 6x6 matrix, with the questions occupying the columns and rows instances. According to John Zachman, Zachman Framework is "a theory of the existence of a structured set of essential components of an object."

The main goal of this study was to analyse the use and interactions among all the stakeholders involved in health systems. And, the Zachman framework conducted the data collection framework facilitating the definition of the specific necessities and the encountered problems of the institution. After the first picture of the institution, data started to be organised and ideas emerged. We needed to frame and to classify the whole tools and documents in order to sort complex problems in information system domain.

Project 3

An anonymous online survey for IT developers was made throughout programming forums, social networks, e.g. LinkedIn. The forum's participants were about developers that worked with multiple Frameworks. This study was conducted through the use of different research methods. Data was collected from staff interviews, observations, and surveys. Interviews were conducted with staff from 11 Portuguese companies whose core business was IT consultancy operating in several target markets with over 5 million Euros. Interviews demanded a preparation process developed beforehand: background information about Companies' Frameworks, if they were collected through books and Internet articles which permitted to understand as well as to describe Frameworks' components and the way they work. The researchers also took in consideration around six years of experience in one of this kind of companies. We met with the respondents in a private room and with they consent. The interviews were audio recorded for later transcription. Based on the script, several questions were asked and in parallel we took notes. Following the literature, different sets of profiles were identified to establish the questions according to interviewees' experience. After face-to-face interviews, we did memo writing, which has high priority in the process of constructing a grounded theory. Memo writing is, according to Charmaz, the pivotal step between data collection and the draft writing process. When an idea is conceived, ''it prompts you to analyse your data and codes early in the research process'' (Charmaz 2006:72). The processes of reading and note taking of frameworks led to creating a spreadsheet with some questions that should be asked to interviewees. It was realised that their different profiles required different forms of questions. A developer, for example, sees the Framework in a different way that a supporter or an architect does. Defined the profiles and questions, meetings were scheduled with all the persons. It was decided to start with some of the researcher's colleagues to get more experience and to help in the questions' redefinition.

Project 4

The survey we used in project 4 had three large groups of questions concerning demographical data usage in music. The goal was to obtain certain orientations for the proposed system functionality to perceive the acceptance rate of the proposed system. The survey was designed and implemented both in English and Portuguese languages. Some of the information to be collected was about people's gender and age, and about prior knowledge of music theory as well as prior musical practice of instruments. We were also interested in understanding if the participants had technical conditions (computer, internet access, smart phone) to use the proposed system. Finally, we wanted to know the tendency for using learning applications and its frequency of use. The survey was made up by 14 questions. The questionnaire was designed for students and or music practitioners. The question structure was preceded by a description. This description summarized the main subject and intentions. The information about the frequency of practice was very important to consider regarding the design of the system, especially for user experience design: if an application is used on a daily basis, users have a very fresh knowledge of the application, and from its current state. When they have a more sporadic practice, it requires additional summarizing tools in order to grant this comfort of almost immediate knowledge of the application state. About the questions regarding the use of technology in the learning process, a great part of the interviewees informed that they recorded themselves either in audio or video. Likewise, most of them used websites and applications to develop their knowledge and skills about new musical techniques. After the survey analysis, we did 10 interviews to professional and amateur musicians. Our goal was to expand some of the knowledge gathered by the questionnaires. Thus, the questions were about learning and practice habits: self-perception, and perception of one-self and ways to increase it.

5. DISCUSSION AND RESULTS

Considering that resilience is about change, and that change brings more change, we present the changes we proposed to improve each an organisation's resilience. Proposals highlight some of the issues reported on the answered questions obtained by the use of the research methods described before. One of the starting points to build resilience was: making connections - we intended to establish good relationships with the staff of each organisation, accepting help and support from them to assist their IT concerns. We made them understand that some changes will occur and that they should accept change as a part of new living (certain goals might no longer be attainable as a result of adverse situations). We presented them the plan for the development of realistic goals and we also asked them to help us taking decisive actions. In some situations we could, together, focus on past experiences to learn about what strategies for building resilience might work better.

Project 1

Table 1 presents a summary of the main issues, intervenient and proposed solutions and opportunities for the studied institution. According to some observations and interviews, it was found that, nowadays, social solidarity institutions do not interact, efficiently and effectively with the social centres for the elderly. Almost all the institutions use different information technologies systems for management, accounting, and for other concerns. Conversely, the employees of these institutions have different backgrounds and experiences and normally they are not motivated to use technology. In situations where they needed to exchange information, a diverse number of forms and documents were exchanged.

Table 1
Project 1 - Intervenient, issues, solutions, opportunities

Project	Intervenient	Issues	Solutions	Opportunities
A Framework for e-government	Elderly people	Information systems disaggregation	New layouts	Improve quality of life services within:
	Children	Support conditions delivery	New functionalities	Leisure centre, activities, day care centre, domiciliary support
	Administrative staff	Different software packages	New application design	...
	Technical staff	Procedures/Interactions	New output documents
	Other	Information management:	Training with IT
	Stakeholders	Access points to consult information	Reduction of paper use
		Information exchange among institutions	Management tool for the communication process
		Complaints	Optimisation of human resources skills
			Software customisation
			Framework design as a guideline for communication
			Interactive application to manage intern information
		...	...

We proposed a framework, which will be a management innovation tool to serve the communication process among the social centres for the elderly and the national social security institution. This attempt will contribute to improve the interaction among all partners and to address organisations' agility and innovation. Before arriving at the proposed solution, we analysed the scope, the business processes, the information systems for elderly care and the available technology for social institutions. We gathered data about the information systems concerning the social system and the SCLF centre services, the technologies, the users, the procedures, and the interactions in and out these institutions.

According to our research, we concluded that, in general, information was spread by different means of communication in a combination of digital and paper forms. The human resources had few appetencies for technology use and were scared for the amount of activities that social security demands.

The main problems encountered in an SCLF centre, with the information systems use, were based on the absence of customization hypothesis, the lower experience and background of the institution's workers, and conversely, on the frequent legislation changes. There were two software programs: one responsible for the accounting management and the other for user management. The former could only be functional for an accountant with some experience. The software had specific features which do not permitted a person who was not able to use accounting concepts, to use it properly. The software was divided into several areas; each one had its particular characteristics. Several features needed constant updates according the new law decrees; and conversely, the compatibility issues with the supervisory bodies were not easy to make. In parallel, with the difficulties using the software, users dealt with information exchange in paper format, which was not aligned with those in the software. The latter, the management software was extremely complex due to its virtual use. Through this software users could create and record all parameters of the institution receipts and they could make the calculations about what each customer spent. A list with all the people on the waiting list could be drawn.

One of the major insights we have reached within this study was that innovation methodologies were implemented across the social centres in interaction with the social security system. Human efforts engaged in the innovation process, tools, and technologies. The requirements, processes and outcomes were clearly defined and staff and employees were involved and motivated in this investment. The communication channels were identified and the structure of the innovation interventions process was defined. We proposed a framework, in parallel with the proposed interactive application; to help as a guideline for communication among the different institutions of social solidarity (PISS) since each one has diverse types of communication to exchange among them.

The innovation introduced in the social centre shown alternative pathways of thinking and acting; the majority never previously explored. Changes like involving employees, staff and other stakeholders in the discussion of the information flow were made. The communication among the information systems was established. And an interactive application for the centres management was implemented. As a result, a set of good practices were recognised and are, presently, followed by the whole intervenient in these institutions. Furthermore, the proposed solution acted as an enabler of organisational agility maximising the gaps of communication in and out the institutions involved.

Project 2

In an organisation that involves considerable amounts of information and many service sectors, there are several information systems that are integrated by technologies capable to do so. This interoperability between different systems avoids misunderstandings and enables data communication between different departments, which is essential in an area that deals with human beings and where any error is serious and calls into question the integrity of human life. However, it is necessary not only to promote the integration of information systems to facilitate communication, but also to manage and monitor these systems.

A classification model in two dimensions, based on questions that address the basic needs of work and communication was used to have a picture of the institution and to characterise the intervenient within the process. It worked as a guide to answer some questions. These questions are What? (In order to collect the set of methods and processes that are of interest in the profession), How? (Set of methods and processes required for the occupation), Where? (Places of interest and relevance), Who? (Community and interests in a given occupation), When? (Temporal obligations and duties, which defines the time schedule and tasks), Why? (Reported reasons from the professional performance of certain tasks). This structure gave us an organised overview of the health centre processes and it permitted to have more information about the players in each process and the technologies to consider.

To answer the questions what, how, where, who, when and why, within the organisation and its needs, a communication link was established between the researcher/programmer, and the whole community involved in it(table 2). We start having answers from health professionals and other stakeholders in a clinical setting. It was essential to find partners with health centre to perform a direct data collection. Within the work that is being developed, direct contact, with patients and health professionals from a health centre was established in order to understand the centre functioning and to understand how people see services.

Table 2
Classification model

	What	How	Where	Who	When	Why
Scope	Description of important health services and information systems.	Data collection, research and contact with users.	Medical centre, with all stakeholders, from health care providers to patients.	Employee, health professionals and patients.	Whenever possible and in situations or important dates.	Impact and importance of health in society and the benefits of technology.
Business (conceptual)	Using technology to promote the health, increase well being of patients and promoting good practices minimising failures or errors.	Finding patterns of operation to define the methodology and actively work with users.	Layout pattern of operation of the centre and all activities.	All direct or indirect participants in clinical functioning: medical, laboratory technicians, radiologists, cardiologists, staff, reception staff, and patients.	Regular visits to the centre in order to establish direct contact.	Evaluate the functioning of the organisation as a whole; contribute to improve the operation, communication and medical care.
Health information system (logical design)	Vocabulary and terminology, attributes detailed, description of the operation of information systems	Data flow diagrams use case models, mythology, architecture s, security models, and models of data transmission and communication flow.	Controlled environment models, Test area Models,	Assigning responsibilities to each employee of the centre and their access to specific components of the health care information systems.	Cycles for clinical study (every week data collection is done, and status)	Purpose of organisation, procedures used, function and size of information systems.
Technology (Physical design)	Communication protocols, syntax, programming language, description of health care information	Systems design, accepted protocols, user manuals, practice guidelines, and authorized activities.	Technical Architecture, network Architectures (Routers, WANs, LANs), laboratory, medical facilities.	Physical health care organisation.	Cycles for study of technology and all surroundings.	Understand details of regulations, defining business strategy and operation of the organisation. Bridging found grace.

The health centre had a considerable amount of technologies in use; however, its use was not in maximum performance. Apparently, those technologies seem to promote good service, which was not enough to provide the best service. There were some interfaces that provided useful information for users and, computers with software, which enabled to manage, enter, and find information more quickly and efficiently. We wanted to understand how to improve the health centre services and the reasons for users' complaints. Stories and experiences have contributed to data analysis. At this stage, ​​an exhaustive search for existing clinical support tools was made. However, we find some difficulties since most of the users are reticent about change and about the use of information technology. Table 3 presents a brief summary of the issues, intervenient, solutions and opportunities we found in the health centre and in interaction with other stakeholders. This project is a work in progress project, which is reason why we cannot arrive to great conclusions. However, from the data already collect we can make the following assumptions:

Table 3
- Project 2 - Intervenient, issues, solutions, opportunities

Project	Intervenient	Issues	Solutions	Opportunities
User Centred Healthcar e Design	Doctors	Difficulties with information systems	Reduction of the number of user's per hour	Improve the health services
	Nurses	Diversity of technologies	Training	Workflow changes
	Administrative staff	Time delay services	Rethinking the pathways of services	Time reduction along the processes
	Technicians	Interactions among all intervenient in the process	Usability tests	Increase the relationships with patients
	IT staff	Complains	...	...
	Patients	...
	Service providers
	Other Stakeholders

In software development, usability is a crucial point, because it focuses on applications, tools, interfaces or system simplicity. It also refers to the efficiency of design and design for a particular purpose. A system design whose usability is considered, will give better reasons for its acceptance. This will be our goal. According to Nielsen (1992) there are some usability goals to consider during the design process: Ease of learning, Efficiency/Effectiveness, Easy to memorise, Errors, Motivation, Utility and Integration. The health centre employees and other health care providers have some problems using software since they do not have the necessary computer skills. So, they will be called to participate in training sessions during the use and development of applications in order to produce something intuitive, with easy navigation, and with task self-explanatory guides. The absence of experience and learning methodologies increase the under-utilization of applications. People require faster applications that can optimise time and reduce waste. Thus, it is important to realise that "lengthy workflows, cumbersome functions, key tasks with jargons are typically what render the execution of tasks slow. Processes and workflows that have optimal turnaround times, quick page loading time, and have simple interfaces that facilitate faster information processing make an efficient EHR application" (Goodman 2011).

Project 3

The research goal of this study was to answer the question about the high levels of user rejection of a requested project. We wanted, also, to understand the reasons for this rejection. After data analysis it was found that the business sectors that most use companies' frameworks were those of Telecommunications and Media, High Tech, Financial Services and Public Administration as shown in 4. About 89% of respondents had worked with a Framework and from these 43% had between 6 and 10 years of experience.

Fig. 4
Frameworks' use by Activity sectors

The benefits for developers, who have worked with the frameworks, were among others the speed of implementation, structuring, reuse and efficiency. The major identified drawbacks were the lack of documentation and the costs of introducing new features as well as versatility of the Framework in accordance with certain customer requirements. For companies, the most obvious advantages were the productivity increase associated with a shorter development time, higher profitability and the creation of standards inside the company. Some key factors to the creation of Frameworks were also identified: training, speed implementation, performance and security.

The average investment by companies on these products was from 10% to 15% of the budget. Table 4 presents the intervenient, issues, solutions and opportunities analysed from the project 3.

Table 4
- Project 3 - Intervenient, issues, solutions and opportunities

Project	Intervenient	Issues	Solutions	Opportunities
Frameworks in Information Technologies Companies	Staff in the company	Provision of products & Services on a reduced time	Identification of the frameworks segment market	Training
	Staff from similar companies	Frameworks as tool of accumulating knowledge & experience	Decision factors for the choice of a framework	Speed implementation
	IT developers	Reused code	Customer 'satisfaction	Performance
	IT supporter	Identification of framework's components	Advantages of frameworks' use:	Security
	IT architects	Profiles identification	Enterprise business	Versatility
		...	Development teams	Adaptability
			Areas of improvement	Introduction of new features
			...	Profitability
				...

From the interviewees' responses, it was considered that the support for multiples systems was a challenge for the future, side by side with the versatility and adaptability, and also the continuous evolution and the introduction of new features in the framework. The companies that were betting on this type of product, a framework, although they were achieving great profitability, they needed to do some more work concerning new solutions to their customers. The chain of intervenient in the process of demanding and accepting a solution should be decreased: there are a great number of intervenient with different backgrounds and experiences involved in the commercial side of the project development. These people also have different roles and interests. We found that this problem should be sorted, probably, by rethinking the number of intervenient in the process.

Project 4

Music amateurs and professionals answered a total of 86 inquiries from 28 June to 10 July 2013. (40%) of the respondents had a profile from 26 to 35 years old, most of them were males (84%), with prior knowledge of music theory (72%), and (86%) had to learn a musical instrument. 39% played string instruments and 25% played the drums or percussion instrument. Almost half (44%) of the respondents practice an instrument more than twice a week, 33% practice every day. (Figure 5 a) and b)).

Figure 5
Instrument practiced and Frequency

Learning in a classroom was considered very important, as well as learning from a teacher. Most of the respondents (>78%) classified learning from a teacher as being very important. Participants avoided responding on extremes concerning distance learning; there is a central tendency on choice 5 (24.42%), and a more or less even dispersion in the other choices, as figure 6 demonstrates. The importance of learning alone presented two trends, one rating 5/10 (23%) and other 10/10 (21%).

Figure 6
Distance Learning

We were interested in understanding three main aspects of music learning: the musical learning process regarding different aspects of learning, the physical aspects of learning and the new musical learning systems. From the analysis made to the data gathered, we concluded that most practitioners found learning alone very important, which is significantly different from distance learning . However, class-based learning, and learning from a teacher assumed a higher importance than learning alone.

Concerning the technical requirements for the use of musical applications, the results were very interesting: 56 % of the respondents had already tested online musical learning systems. In respect to musical learning applications, 53% of the participants had not used it and within these, 35% used it more than twice a week. Most of the inquired (97%) never attended an online music course. The main reasons were the inexistence of technical means to its attendance, the credibility of such courses, the lack of evaluation periods, and consecutive crediting of its students. The most important result was the interest in a musical learning application: 82% demonstrated interest in a musical learning application, from which 89% would allow biometric data measurement for the purpose of their learning statistics.

Table 5
Learning tool for Musicians

Project	Intervenient	Issues	Solutions	Opportunities
Learning tool for Musicians	Students	Development of a system to teach music practice	Development of the new applications	A new application for an interactive musical learning
	Music practitioners:	Acceptance rate of the new system	Consider the whole types of learning (alone, with a teacher, at distance)	Inclusion of biometric data for statistical purposes
	Professionals	Frequency of the instrument practice	...	...
	Amateurs	Needs and motivations for learning
		Learning from a teacher vs learning alone
		Class based learning vs distance learning
		...

5.1 Constructive Actions to Support Resilience

In general, the use of the presented research methods permitted to understand the requests and needs of the inquired people. The lime survey resource was effective to get the necessary data for the application development. The concerns of the target user contributed to structuring the inquiry and easily managing the data. However, some of the data analysis brought new doubts to the study (project 4) leading to new questions that need to be analysed in depth. We need, for example, to deeply understand the respondents resistance to online learning. This question is very important for the development of the musical learning application. The variables Distance Learning / Learning from a teacher should be distinguished in a clear way. Probably, we should question if there can be a coexistence of both, i.e. distance learning but having a teacher, physically or not, to complement the learning process.

The main findings we got from project 3 were: some activity sectors invest more in frameworks than other sectors, like telecommunications. A great number of users gave a high score to some key attributes about the framework's use and their advantages: reduction of time development and the increase of profitability. The two of the more evident concerns are the lack of documentation and training, on these areas; companies need to do some effort to improve the quality of utilisation of the framework.

The main results of project 1 were the acceptance of changes considering that a new system with new functionalities was developed; the reduction of paper use; the aggregation of information systems; the human resources optimisation - some employees change workplaces to increase their satisfaction and to optimise their skills; software customisation.

The output from project 2, according to the development of the search, was the proposal of patients' reduction by doctors per hour. This will decrease the time delay complaints. Doctors who were making diagnostic exams will will no longer do it and, nurses will do those exams. Administrative employees will have IT training to improve their knowledge. Information workflow will be optimised since several non-necessary tasks were identified and they will come to an end.

The research methods were the tool to support resilience. In parallel, technology becomes also an instrument to carry out resilience strategies until each proposed solution is offered. It was interesting to observe that, at the end of the process, people used technology to restore collaborations while there was a key environmental interference at its beginning, since some of them did not have motivations and skills. The intervenient figured out the role that technology played in providing them with alternative resources to reconstruct, modify, and develop new routines, or patterns of action, for the work and even for their socialisation.

After all, we considered that resilience can be found either in the used research methods, or on the solutions we proposed and also in the people that intervene in the process. Individual or group resilience was built through the satisfaction people seemed to have with the proposed solutions: through the analysis of the who, what, where, when, questions; through the self-awareness increase by the identification of their own strengths; the planning, steps definitions, and activities lists were important to give us patterns to figure out the proposed solutions. Conversely, the used constructive way to respond to each of the involved community gave us self-confidence and contributed to reach an efficient way to propose realistic solutions.

These different projects contributed to improving an organisation's resilience, being resilience the ability to recover from negative experiences and becoming stronger with the new project proposals. Different organisations had different needs. The presented projects were outputs developed to achieve the resilience of organisations improving their unstable environment and helping to perform effectively.

6. CONCLUSIONS

This paper presents the use of mixed methods in human-computer interaction field. Focus group, surveys and interviews are generally applied with social sciences research. Our goal was to exemplify them within the HCI field, where researchers are focused on producing artefacts or services to be efficient and to satisfy the target user. The development of applications would not be effective without the target audience identification and its participation before, during and after the project design process steps.

The term resilience has been applied to everything from the real estate, to the economy, sports, events, businesses, psychology, and more. The intention of this study, with different projects, was to firstly produce as much information as we could, by using mixed methods research, and then, analyse the existing information systems and technologies to develop solutions in order to impose resilience on either employees training or user performance improvement.

In the presented projects, we found that the methods contributed to starting, changing or complementing the researcher's initial ideas: in project 4, after the data analysis, it was possible to redesign our first application attempt and to integrate the users' suggestions. The results of usability and reliability tests, as well as, the survey results permitted the development of the working prototype following a user centred design approach. Therefore, the future work will take into account most of the gathered data, namely, some of the features referred to by the respondents and interviewees. Concerns about privacy settings will be customisable as a user's preoccupation. The survey available to frameworks' users, within information systems companies, produced some information concerning failures, which would be considered to improve the success of the frameworks. The data obtained will be used for companies in the area of information systems to improve their performance. In project 1 we had very good staff participation in our data analysis and also in the process of solutions proposal. Project 2, is in progress, however we are satisfied with the information we got and the main proposals we did to change workflows, software optimisation and time delay reduction in patients being served.

A great challenge within these projects was the understanding about the kinds of events that have been most stressful for the whole intervenient, and those events that typically affected the organisation. We concluded that, in general, people are afraid of any changes, either mobility from and work places or changes in routine tasks. The staff was not really concerned with financial questions, they were concerned with how changes will interrupt the organisation's normal life. Also, employees were ashamed of their skills when demanded to have IT training. Stakeholders, in general, were more open to change than the organisations' internal staff.

In this study, in practice, some resilience components were highlighted, such as participants' different skills and abilities (self-awareness, creating meaning from other experiences, self-efficacy, optimism, and building strong relationships) which contributed to a more or less efficient way to face the proposed changes in each organisation, as well as, to enhancing organisation's resilience by the use of different strategies.

The Human Computer Interaction design approach applied to user technological interfaces design, using different research methods, contributed to satisfying both parts: the user and the organisation. The output was mainly the promotion of the use of knowledge and methods for users in particular, and for the organisation, in general; the understanding of guidelines and models, to solve encountered problems and, the technology analyses of people in both individuals and organisational contexts.

• Published by/ Publicado por: TECSI FEA USP - 2016 All rights reserved.

Acknowledgments

This work has been supported by FCT - Fundação para a Ciência e Tecnologia within the Project Scope UID/CEC/00319/2013. We acknowledge the participants in each of the presented projects.

REFERENCES

• Beinhocker, E. (1999), Robust adaptive strategies. Sloan Management Review, Vol. 40(3): 95-106.
• Bhamra, R., Dani, S., & Burnard, K. (2011). Resilience: the concept, a literature review and future directions. International Journal of Production Research, 49(18), 5375 - 5393.
• Boyes, H. Resilience and Cyber Security Technology in the Built Enviroment, Published by the Institute of Engineering and Technology, London, UK.
• Charmaz (2006), Constructing Grounded Theory: A Practical Guide through Qualitative Analysis, SAGE Publications
• Cooper, Alan et al. (2007), About Face 3: The Essentials of Interaction Design Indianapolis, Indiana: Wiley. p. 610. Retrieved 18 Jan 2015.
• Coutu, D. (2002), How resiliency works. Harvard Business Review, Vol. 80(5): 46-55.
• Creswell, J. (2014), A Concise Introduction to Mixed Methods Research (Sage Mixed Methods Research), Sage Publications, Inc
• Deevy, E. (1995), Creating the resilient organisation. Englewood Cliffs, NJ: Prentice Hall.
• Denis, Col, Rensel, J. (2015), Resilience - A Concept, A Publication of the Defense Acquisition University, Defense ARJ, Vol 22 N3: 294-324
• Eklund, Tomas, et al., (2008), Usability Evaluation of an XP Product, in Proceedings of the 19^th Australasian Conference on Information Systems (ACIS), 280-289
• Freeman, S., Hirschhorn, L. & Maltz, M. (2004), The power of moral purpose: Sandler-O'Neill & Partners in the aftermath of September 11th, 2001. Organisation Development Journal, Vol. 22(4): 69-81.
• Goodman Elizabeth, Stolterman Erik, Wakkary Ron, (2011), ACM Human Factors in Computing Systems: Understanding interaction Design Practices
• Hind, P., Frost, M., & Rowley, S. (1996), The resiliency audit and the psychological contract. Journal of Manage- rial Psychology, Vol. 11(7): 18-30.
• Holling, C.S., & Gunderson, L. (2002), Resiliency and adaptive cycles. In Lance Gunderson and C.S. Holling (eds.), Panarchy: Understanding transformations in human and natural systems, 25-62. London: Island Press.
• Hollnagel, Erik, (2013), Resilience Engineering in Practice, Edited by Erik Hollnagel, MINES ParisTech, France, Jean Pariès, Dédale SA, France, David Woods, Ohio State University, USA and John Wreathall, John Wreathall & Co., USA, Series: Ashgate Studies in Resilience Engineering
• Hirschhorn, L., & Gilmore, T. (1992), The new boundaries of the 'boundaryless' organisation. Harvard Business Review, 104-115.
• Hollnagel, E. Woods, D, Leveson, N. (2006), Resilience Engineering: Concepts and Precepts, Ashgate Publishing, Aldershot Hampshire, UK.
• Erik Hollnagel. How Resilient Is Your Organisation? An Introduction to the Resilience Anal- ysis Grid (RAG). Sustainable Transformation: Building a Resilient Organisation, May 2010, Toronto, Canada.
• IBM, (1997), Building Object-Oriented, Retrieved 13 Oct 2014, from http://lhcb-comp.web.cern.ch/lhcb-comp/Components/postscript/buildingoo.pdf
  » http://lhcb-comp.web.cern.ch/lhcb-comp/Components/postscript/buildingoo.pdf
• Laurel, Brenda, (2003), Design Research, MIT Press
• Lengnick-Hall, C.A., Beck, T. E., & Lengnick-Hall, M. L. (2011), Developing a capacity for organisational resilience through strategic human resources management. Human Resource Management Review, 21(3), 243-255.
• Limesurvey retrieved from http://www.limesurvey.com/ on 10-12-2014
  » http://www.limesurvey.com/
• Lowgren, Jonas (2014), Interaction Design - brief intro In: Soegaard, Mads and Dam, Rikke Friis (eds.). "The Encyclopedia of Human-Computer Interaction, 2nd Ed.". Aarhus, Denmark: The Interaction Design Foundation. Retrieved from https://www.interaction-design.org/encyclopedia/interaction_design.html on 10-01-15
  » https://www.interaction-design.org/encyclopedia/interaction_design.html
• Mao, Ji-Ye, et al., (2005) The State of User-centered Design Practice, Journal: Communications of the ACM, vol. 48, no. 3, pp. 105-109
• McManus, S., Seville, E., Vargo, J., and Brunsdon, D. (2008). "Facilitated Process for Improving Organisational Resilience". Natural Hazards Review, 9(2), 81-90
• Nielsen, Jakob, (1992), Finding usability problems through heuristic evaluation. Proceedings of the SIGCHI conference on human factors in computing systems. Monterey, CA, USA, 1992. 373-380
• Norman, Don, (2013), The Design of Everyday Things, Revised Edition, Basic Books
• Riegert, G. Gertman, D. McQueen, M. (2009), Resilience Control Systems: Next Generation Design Research, HIS, Catania, Italy
• Ruttner, M. (1990), Psychological resilience and protective mechanisms. In Jon Rolf, Ann S. Masten, Dante Cicchetti, Keith H. Nuechterlein, and Sheldon Weintraub (eds.), Risk and protective factors in the development of psychopa- thology, 181-214. UK: Cambridge University Press.
• Saffer, Dan, (2011), Designing for Interaction: Creating Innovative Applications and Devices, 2nd Edition,New Riders Publishers.
• Sheffi, Y. (2005), The resilient enterprise: Overcoming vulnerability for competitive advantage. Boston: MIT Press.
• Young, Indi, (2008) Mental Models: Aligning Design Strategy with Human Behavior, New York: Rosenfeld Media.
• Wright, G. Schaetzel, T. Cyber Security: Designing and Maintaining Resilience, White paper presented by Georgia Tech Research Institute, Cyber Technology and Information Security Laboratory, Retrieved from The Zachman Framework^TM: The Official Concise Definition, http://test.zachmaninternational.com/index.php/the-zachman-framework
  » http://test.zachmaninternational.com/index.php/the-zachman-framework
• The Zachman Framework™: The Official Concise Definition, http://test.zachmaninternational.com/index.php/the-zachman-framework
  » http://test.zachmaninternational.com/index.php/the-zachman-framework
• Zimmerman, J., Forlizzi, J. Evenson, S. (2007), Research Through Design as a Method for Interaction Design Research in HCI, Human-Computer Interaction Institute, Carnegie Mellon University

Publication Dates

• Publication in this collection
  Sep-Dec 2016

History

• Received
  29 Sept 2015
• Accepted
  29 July 2016