Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Open Access

LIFANA – toward developing a meal recommender system as a dietary support app for the elderly

Evaluation in field trials, pitfalls and lessons learned, and the path forward

Published Online:October 25, 2023https://doi.org/10.1024/0300-9831/a000795

Abstract

Abstract:Background: Though a variety of eHealth/mHealth dietary solutions exist, many are ill-adapted to the target population and local eating habits. A specific need exists for the elderly, a growing vulnerable population with limited digital literacy. The LIFANA project aimed at developing a mobile nutrition solution, i.e. a dietary meal-recommender app for personalized meal planning useful for the elderly. Methods: In addition to considering age, gender, and physical activity, the app assured sufficient intake of calories and proteins. The solution was optimized to consider local eating culture in Portugal (PT)/The Netherlands (NL) where it was tested. Recipes (>300) were included and aligned with national food composition dietary databases (FCDBs) to analyse their nutritional values for meal planning. Individual dietary preferences, food restrictions (i.e., allergies), and budget considerations were included in the user profile. The development process involved user integration, including focus groups and usability evaluations, followed by longer field trials in Portugal (n=53 participants, age 60–81 y, 14 months) and the Netherlands (n=107, age 52–86 y, 3 months). Endpoints regarding acceptance/usage frequency, anthropometric measures and (in PT) blood pressure and body fat were collected. Results: 23/34 elderly finalized the trials in PT/NL. No significant changes in anthropometry or other assessed markers, including blood pressure, were observed. 9% (NL) and 47% (PT) of users reported that they would consider using the solution if it were on the market. Conclusions: Via an iterative adaptive process, a dietary app was developed and improved that demonstrated acceptance/user-friendliness comparable to other tools available on the market and allowed – despite the COVID crisis – for stable anthropometric markers and blood pressure. However, it was also observed that additional features, such as a link to an online shopping app, and closer personal follow-up was associated with increased usability and acceptance of the solution and thus further personalization and nudges are warranted to increase employment of such dietary apps.

Introduction

A healthy diet is a central pillar among life-style factors and related to the prevention of several chronic, non-communicable diseases, including type 2 diabetes [1], cardiovascular diseases [2], and several types of cancer [3]. An especially vulnerable group of the population are the elderly, a segment which is also steadily growing, and currently encompassing around 1 billion persons worldwide [4]. Following retirement and a potential reduction of physical activity, caloric needs decrease, though the needs of nutrients including vitamins and minerals are even in part increasing (such as for vitamin D), requiring a diet of high nutrient density [5]. In addition, a diet high in protein is advised, in order to counterbalance the risk of sarcopenia, including sarcopenic obesity, which is a major diet-related complication in the elderly [6].

Though many dietary recommendations exist and have been published, the major problem is to achieve a high adherence to such guidelines. A novel approach to reach a large fraction of the population and an attempt to complement professional advice in a cost-efficient way are e-Health/m-Health solutions [7]. Though a large number of dietary apps exist on the market (several 10.000), and such tools have been judged as overall effective [8], also for the elderly [9], many fall short to accommodate the special needs of the elderly, also in sight of the typically more limited digital literacy compared to the younger generation [10]. For instance, in a recent systematic review, the following aspects were mentioned to constitute main barriers of acceptance [11]: i) need for time-consuming logging and monitoring, ii) being unsuitable for some types of populations, iii) overlooking context such as socio-economic situation, among other.

Another recent systematic review examining the number of dietary apps for the elderly available on the market [12] reported that these were generally of moderate quality, as evaluated by the mobile application rating scale (MARS) [13], and shortcomings including information quality, evidence-based approach and also data protection were mentioned, emphasizing a relative lack of suitable applications. A number of aspects should thus be thoroughly considered when designing nutrition solutions for the elderly. Compared to the majority of apps, requiring a relatively large amount of time for logging and/or scanning of consumed food items, meal recommender systems are more user-friendly, though with the disadvantage that they do not completely follow-up and register dietary patterns. This can be considered an advantage when targeting the elderly, who on average spend less time online and have a lower digital literacy [14]. Meal recommender systems have been reported in the literature [15, 16], though shortcomings such as ill-adapted ingredients, too static recommendations over time that do not consider changing user preferences, missing visual aids for choosing, or only recommending frequently-rated food items, have been reported. Furthermore, in a recent systematic review of recommender systems, lack of clear target group definition and lack of graphical aspects to present recommendations to users have been emphasized [15, 17]. Another recent review on food recommender systems for persons with diabetes has emphasized that such tools should better incorporate user preferences, explanation capabilities, as well as a more knowledge/science-driven solution approach [18].

Another important criterion for developing a dietary app is the appropriateness to consider local dietary eating patterns, including typical dishes that are consumed for the various meals during a day (i.e. breakfast, lunch, dinner). This may differ considerably between countries, including European ones. For instance, snacking habits and intake of energy from snacks appear more prominent in northern and mid-European countries compared to the south, also variability of lunch calories was reported to vary considerably by country [19]. Furthermore, local dishes play an important role, owing to the cultural diversity in many countries of Europe, which may be very specific to their country of origin – just consider black pudding in England for breakfast, or frog-legs in France. In addition, as provenience and way of processing could differ between countries, local food composition databases (FCDBs) should be applied when translating recommended food items and dishes/recipes into energy, macro- and micronutrient intakes [20].

Thus, the aim of the present study and this article was to present the challenges and pitfalls toward developing a dietary app for the elderly across Europe, to present solutions overcoming some of the previous limitations of meal-recommender solutions that mounted in the development of the LIFANA nutrition solution, and to demonstrate in proof-of-concept field trials the usefulness of the meal recommender system for the elderly.

Materials and methods

Overall strategy and goals

The overall strategy of the LIFANA consortium was to support users during all phases of ageing – from active seniors to sedentary elderly users and patients in need of daily care (Figure 1), with the goal to deliver to elderly a user-friendly meal recommender that would enable them to follow a balanced diet that would not compromise health and weight status.

Figure 1 The LIFANA Nutrition solution supports different phases of ageing – from active seniors to sedentary elderly users and patients in need of daily care.

To address the special needs of elderly regarding usability and local recipe content, we implemented a User-Centered Design (UCD) process with two iterative cycles of three phases, according to the guidelines published by AAL [21]: i) understanding, ii) conceptualizing, and iii) testing. To facilitate understanding of the user’s needs, multiple stakeholders participated in focus groups and co-creation workshops. In the conceptualization phase, requirements for the LIFANA solution were defined. The LIFANA solution was finally tested in field trials in the Netherlands and in Portugal, assessing usabilty aspect for the targeted age group, such as screen design and in-app navigation, as well as changes of health-related aspects. For the trials in the Netherlands (NL), 300 Dutch recipes were integrated in the LIFANA database for the meal planner, and their nutritional values were estimated (see “Dietary meal recommender app” section). In Portugal (PT), existing recipes from the previous EU project (CordonGris) were used that have been specifically created for the target group by a nutritionist.

The LIFANA solution

For active users (60+) the solution contained the meal recommender app, though integrated physical activity level measurement to balance caloric intake using the GoLive [22] wearable clip, provided by the Dutch project partner Gociety Solutions, for participants of the field trials in the Netherlands. Aimed towards users with limited mobility, the LIFANA prototype also integrated a digital shopping list with grocery home-delivery during the field trials in Portugal (the city of Porto), provided by the Portuguese retail business partner MC Sonae [23] to streamline the entire process of planning meals and acquiring food products. Considering the food retail area, the LIFANA solution could be provided to customers as part of their existing apps, such as the Continente SIGA app [24], to facilitate and promote the practice of a healthy and balanced diet, adapted to the specific needs of the elderly user.

Dietary meal recommender app

The overall aim of developing the app was to provide elderly with regular balanced meal suggestions, personalized for their age, gender, physical activity and personal food restrictions and preferences, considering overall calorie and protein targets. The app was developed by the project partner Fraunhofer AICOS, based on previous work in the projects Sous Chef [25] and CordonGris (AAL Programme, 2016–2017) [26].

Meal-planning: The LIFANA app was developed and made available for participants of the trials by personal invitation for both Android (Google Play store) and iOS (Apple TestFlight) smartphones. In order to request a meal plan, the user first had to specify age, gender, height and weight (body mass index, BMI), and activity level on a self-reported scale (lowest, low, medium, or high). These settings could be changed at any time, but were not prompted regularly. The activity level was translated internally to appropriate metabolic equivalent (MET) factors, so that the daily caloric goal could be automatically set by the LIFANA app based on activity and BMI, according to World Health Organization (WHO) recommendations. The protein goal was set according to the general recommendations for macronutrient distribution (0.8 g/kg body weight, [27]). The user further had the possibility to fill in restrictions, i.e. food items or ingredients that were not preferred for recommendation. The user could choose from pre-defined profiles (e.g. vegetarian diet). Once the personal user profile was completed, the user could request a meal plan (Figure 2) for a single day or for a whole week, which allowed the recommender system to achieve a better balance and variety of ingredients between the days. While the meal recommender proposed adequate options for each meal, users could manually replace each recommendation from a list of alternatives with similar nutritional value. Furthermore, the user could choose which meals of the day to include in the plan (breakfast, morning snack, lunch, afternoon snack, and dinner) and report their estimated physical activity level to adapt the caloric goal to their individual lifestyle. For each planned meal, a picture of the meal was shown, nutritional information could be consulted, step-by-step cooking instructions were presented, and the required ingredients could be added to a digital shopping list for more convenience. For the Portuguese field trial (city of Porto), this shopping list could be sent directly to the Continente app to benefit from price information and their home delivery services. The ingredients of the recipes were automatically mapped to products from the Continente database (product price and availability information is published by the LENGOW e-commerce framework almost real-time), and users could send their shopping list directly to the Continente app to finalize the purchase online.

Figure 2 The main screen gives access to te meal planner and shopping list, and provides a dashboard view of burned calories based on the activity level measured by the GoLive clip wearable device (2a). The meal plan for each day of the week can be accessed by a calender view (2b). Images © Associação Fraunhofer Portugal Research.

Estimating nutritional values: The recipe content for the Dutch database was mostly provided by the Voedingscentrum (The Hague, Netherlands), representing a healthy and modern style of food items, inspired by international cuisine. The content of the Portuguese database was created by a nutritionist from the previous project CordonGris [28]. Based on the nutrient levels of the ingredients, the nutrient content of multi-ingredient foods could be calculated based on Food Composition databases (FCDBs). Due to the regional differences regarding the type and production of foods, we used NEVO in The Netherlands [29]and the Tabela de Composição dos Alimentos in Portugal [30], all licensed from the EuroFIR organisation that provides unified access to the digital versions. Due to the large number of ingredients involved in our recipe database, we automated the process of importing and referencing ingredients with FCDBs. In [31], we reported on experiments regarding our task to align national FCDBs based on semantic food descriptions. Despite the reliable data on the quantity and quality of nutrients and other components, there is no harmonized and standardized method to use those data considering the recipes. Recipes often use “kitchen units”, such as spoons and cups, and there are no globally accepted definitions for the conversion into grams or litres. Similarly, recipes mostly specify the quantity of fruits in units rather than grams, thus additional databases such as Portie-online [32] are required. In order to account for further changes during processing, such as frying and cooking, we applied weight yield and nutrient retention factors for various foods and cooking methods according to Bognár [33].

Underlying software architecture: The LIFANA solution was divided into a frontend component that implemented the user interface as a mobile application on a smartphone, and a backend (cloud server) one that provided recipe databases (one for each country), recipe authoring tools, and the meal-planning algorithm. Furthermore, an optional wearable fitness tracking device could connect with the smartphone via Bluetooth to provide activity level measurements. The meal planner was implemented as a software service that delivered a meal plan, taking personal information (age, sex, BMI, food restrictions, activity level), caloric goal, and recipe database into account as inputs. All recipes’ ingredients referred to FCDB food item identifiers so that the meal planner could filter recipes with ingredients that matched user-specified restrictions or pre-defined diet profiles, e.g. a vegetarian diet. Furthermore, all recipes were filtered by their specified suitability for different meals (breakfast, lunch, dinner) and divisions (cereals, fruits, main dishes, soups, desserts). In addition, recipes could be rated by the user on a five-point scale to appear more or less frequently in the plan. The recommender algorithm employed a content-based approach and information retrieval techniques. It applied a set of rules and heuristics to select recipes that fulfil the caloric needs and enable dietary diversity. The algorithm did not implement other popular techniques, such as Collaborative Filtering or Machine Learning, which could further improve the quality of the plans in the future.

App usage analytics: To better understand how people were using LIFANA, we integrated Firebase Analytics into the mobile app, with an internal tool for data visualization from the field trials such as meals plans created or preference data. Through Firebase Analytics, we collected data about the active users and their engagement with specific features of the application, such as (for Portugal) adding products to the shopping list, removing products, exporting the shopping list to the Continente app, and export by email. A platform for data visualization developed by Fraunhofer Portugal was used to analyze data stored in the LIFANA database, such as meal plans created, preference data, or survey data.

Focus groups

Focus groups constituted the first cycle toward a further iterative improvement of the LIFANA solution. The project implemented a User-Centered Design process with two iterative cycles of three phases: understanding, conceptualizing, and testing [21]. To facilitate understanding the user’s needs, 17 seniors aged 80+, health-care professionals and informal caregivers participated in a total of 3 focus groups and co-creation workshops. For this purpose, half-day meetings were set up in the Netherlands (1) and in Portugal (2). These included in-depth interview with participants and health-care professionals, as well as co-creation workshops. The latter contained speed-dates, mind-maps/collages and round table discussions and comment-collection.

Personas and scenarios were defined to represent the findings from the first phase. In the conceptualization phase, requirements for the LIFANA solution were defined and ranked by all partners.

Usability evaluation

The testing phase started with usability evaluation studies, where 12 users participated in testing the screen design of the alpha-version prototype in three iterations (7 in PT and 5 in NL), and in the 4th iteration 5 joined in for testing the beta-version (PT). According to Faulkner [34], it can be assumed that our sample size of 17 users identified more than 90% of usability problems (97% on average). Participants were recruited through existing networks in each country. The first two sessions took place in Portugal, the third session in the Netherlands. The participants in PT were recruited by Santa Casa and Fraunhofer AICOS through its internal recruitment network COLABORAR, while in the NL were recruited by KBO PCOB. Participants were required to be over 55 and to be able to use a smartphone on their own.

The testing material consisted of a functional prototype of the LIFANA mobile application on a Samsung smartphone with an Android operating system. Different iterations of the prototype were evaluated in the different usability testing sessions. Individual sessions took approximately between 10 and 20 minutes and were carried out by a single researcher on the first two iterations, and by two researchers on the third and fourth iteration. Sessions were video recorded with the camera, capturing the interaction of the participant with the application. A characterization questionnaire was performed at the beginning of the session regarding age, gender, and smartphone experience (type of phone, number of years) (Table 1).

Table 1 Four iterations of usability evaluation: participants’ characterization questionnaire and task performance. Per iteration, all participants were asked to solve each task and the result was rated either as success, partial success, or failure. The shown outcomes indicate for each result the percentage of participants averaged over all tasks

During the evaluation, the design team asked participants to solve some typical tasks, e.g. create a meal plan for tomorrow without breakfast, and noted any observed problems, to ensure that the final screen design (font sizes, colors, labels, icons, etc.) and menu structure was appropriate for the target group of elderly users. For each task, completion result (either success, partial success, or failure) and error rate (deviations from ideal path) were measured. The number of tasks per iteration and the averaged results are listed in Table 1. After the completing the task, participants were invited to fill in a short questionnaire about their experience with the application. To give some practical examples, the following observations were reported to the software developers in the first evaluation to improve the usability:

  • –
    The Android system “back” button function was unknown for participants without experience with smartphones. Recommendation: Changing the native interface would potentially create more problems for participants who already know this pattern.
  • –
    Buttons positioned at the bottom of the screen go more easily unnoticed. Recommendation: Moving these buttons to a more central location could bring them closer to users’ field of view.
  • –
    Some views included a list with checkboxes that users could use to make a selection of list items, and then apply an action to that selection. While this is a quite common design pattern, it was not clear to all participants. Recommendation: Remove the selection from the shopping list view, and let the user make the selection after selecting the action.

Long-term field trials

The general aim of the field trials was to study the effect of using the LIFANA solution in the general, healthy elderly population longitudinally over time, regarding acceptance of the solution and to monitor their weight as primary outcomes.

Trials in the Netherlands

Overview: The field trials in the Netherlands were organized by KBO (Harlem, The Netherlands). Toward this end, several consecutive field trials were carried out to allow for further iterative technical improvements of the LIFANA solution. In the end, 3 trials were carried out. A first one was discontinued after 3 months due to high drop-out rate, owing to the perceived non-technical readiness of the product. A second trial that was run for 3 months was then ended in order to allow for further improvements of the LIFANA solution. A third and final trial of 3 months length ensued.

Participants and recruitment: Participants for the field trials were recruited starting from March 2019 through (social) media channels of the association KBO-PCOB and its local divisions and through the website of the NVD (Dutch Association of Dieticians). Potential participants could sign up for the field trials through filling in an online form, or by phone by calling the association. Eligibility criteria included living in the Netherlands, being at least 60 years of age but no older than 85 years, while exclusion criteria were having morbid obesity (BMI>35 kg/m2), being underweight (BMI<18.5 kg/m2), having a severe physical or mental handicap, having cancer or any other chronic metabolic diseases, being institutionalized, following a strict diet, or not speaking Dutch.

Intervention: The components of the intervention included the LIFANA app for Android and iOS smartphones. Users in NL also received the GoLive Clip wearable device (Gociety Solutions BV., Leende, The Netherlands) to support activity level tracking. The additional GoLivePhone app was installed to receive data from the clip by Bluetooth. However, it was not mandatory to wear the clip and the activity level could be self-reported, so the impact of the physical activity on the meal plan could vary from participant to participant. If participants took off the device for some period, that data was lost.

The participants were interviewed at time =0 and at the end of the trials. The field trials’ main outcomes were “usefulness and satisfaction with the system” and “technical reliability”, which were assessed by questionnaires, and body weight change over time, which was self-reported. A general life-style questionnaire assessed physical activity, lifestyle and (mental) health, need of care, attitude and knowledge towards technology, expectations towards LIFANA and financial situation, demographic information. The questionnaires were self-administered. Throughout all trials, the participants were able to contact the trial organizers when experiencing any difficulties. Contact could be made by phone or e-mail. Additional contact moments, initiated by the trial organizers, were created by phone or e-mail. Screenshots to support feedback were gathered to understand the problems that participants experienced.

Acceptance of the LIFANA solution: To assess the acceptance of the LIFANA solution and how this changed over time in the participants, subjects in trial no. 3 (n=23 persons) were asked 6 questions at the beginning and at the end of the 3 month-trial. The following questions were asked: 1. LIFANA will help me/helped me to improve my eating behaviour. 2. LIFANA will help me/helped me to make a better food planning. 3. Will have/didn’t have many problems using LIFANA. 4. Using LIFANA will improve/improved my weight. 5. After using LIFANA > 3 months I will know/know more about healthy food & exercise. 6. I will certainly continue using LIFANA after the end of the trial. Possible answers to the questions were: 1 – fully agree, 2 – agree, 3 – neither agree nor disagree, 4 – disagree, 5 – strongly disagree. Thus, a high rating would rather reflect disagreement.

Recruitment and drop-outs trial 1: Participants for the first trial were recruited between March and July 2019. A total of 151 people did sign up. As LIFANA was still undergoing technical development and participants would undergo challenges using the app, people with IT experience were preferred to start with the test. A total of 27 people were selected and invited to a personal meeting followed by the kick-off meeting. The purpose of the kick-off meeting was to introduce the LIFANA Nutrition app, to perform anthropometrical measurements, to install LIFANA, and to create a meal plan with the LIFANA app. First impressions and experiences with LIFANA were noted. Three people did not start due to different reasons (sickness, vacation). Nine people were excluded after an informative conversation concerning the project (lack of IT knowledge, regular drug intake, unfortunate timing of testing period, sickness, or no interest after receiving further information). A control group who did not receive the LIFANA solution was also set up (13 persons). However, due to technical difficulties of the LIFANA solution and the resulting large number of dropouts (>20), the trial was discontinued.

Recruitment and drop-outs trial 2: The second field trial was carried out between September and November 2019. To increase adherence to the trial, a WhatsApp group was formed in which participants could be informed on updates concerning the app and could support one another. Out of the 16 persons recruited, one dropped out for personal reasons (Figure 3), and 9 persons delivered a complete dataset at the end, including BMI.

Figure 3 Participants and complete datasets/entries obtained from trial 2 and 3 in the Netherlands.

Recruitment and drop-outs trial 3: The third trial was carried out following further updating of the LIFANA solution and addressing reported problems in trials 1 and 2. Functionalities such as food allergies and vegetarian/vegan dietary patterns were regarded as an important addition to the app. In May 2020, an online questionnaire about nutrition and lifestyle was created and filled in by Dutch seniors. At the end of the questionnaire, respondents gained information on the LIFANA project and were asked to participate in the LIFANA trial. The participation included four online questionnaires within a period of 3 months that did take place between June and September 2020. Seniors (n=136) replied for this call who gave 83 complete entries (Figure 3), though only 43 participants remained within the study until the end, resulting in a final drop-out rate of 68%. The characteristics of the persons who finalized this trial are given in Table 2. The participants were well educated and had in general a good income of 36.000 Euros per year.

Table 2 Overview of participants of trials in the Netherlands who finalized trial 3 (month 3, n=23)

Trials in Portugal

Participants and recruitment: Field trials were organized by Santa Casa da Misericórdia do Porto (SCMP, Porto, Portugal). Participants for the 14 months trial were recruited between April and October 2019. Former SCMP’s employees were contacted via email with an attached flyer and an online form and through SCMP’s Facebook page. Also, the SCMP team visited several organizations, such as senior universities, pensioner’s associations and the sports faculty of the University of Porto, which organized senior classes to promote physical activity. Potential participants could sign up for field trials through filling in an online form or applying in person. Eligibility criteria included living in Portugal, aged 60–85 years, with normal weight or being overweight (BMI<30 kg/m2). Exclusion criteria were having obesity (BMI>30 kg/m2), being underweight (BM<18.5 kg/m2), having a severe physical or mental handicap, having cancer or chronic metabolic diseases, being institutionalized, following a strict diet, not speaking Portuguese.

Intervention: Participants in PT used the same LIFANA application on their smartphones as in NL, but with self-reported activity level instead of the GoLive wearable device. Observed outcomes were changes over time (before-after) of the targeted parameters. Blood pressure (systolic and diastolic) and anthropometric measurements, including height, weight, body fat, waist-hip circumferences, and BMI were measured during the three time points (T=0, T=9, T=14 months, Figure 4) by trained staff. The height was measured by a stadiometer, weight, BMI and percentage body fat and percentage body water by a bio-impedance scale (Seca 804, Seca, Hamburg, Germany). Waist and hip circumferences were measured with a measuring tape by the staff of the SCMP, blood pressure was measured with the participant seated and after a pause, repeating the measurement three times and averaging the results.

Figure 4 Overview of recruitment scheme, dropouts and number of collected datasets of the field trial in Portugal.

Acceptance of the LIFANA solution: A set of questions regarding technical acceptance of the LIFANA solution were asked at each time-point (Table 3). These were measured at 2 time points, comparing subjects at baseline (n=33) vs. time T=14 months. A set of separate questions was developed to assess the suitability of the Continente application (Table 3).

Table 3 Questions and outcomes related to using the LIFANA solution in Portugal based on 33 participants. Timepoint 1= onset of trial, timepoint 2= at the end of the trial (month 14). Participants were able to choose from the following answers: 1. Fully agree. 2. Agree. 3. Neither agree nor disagree. 4. Disagree. 5. Fully disagree. Values shown represent the average answer of participants

COVID-19 related questions: In order to assess whether the COVID-19 crisis did considerably modify individual’s behavior related to cooking, eating and physical activity, a questionnaire (Table 4) was sent out at the end of the 14-month field-trial to the 33 participants from whom also BMI data and LIFANA-related questions were collected from. Times of strict lockdown in Portugal were: 18/03/2020–02/05/2020 and 04/11/2020–30/04/2021.

Table 4 Results related to COVID-19 and altered cooking behavior, perceived stress, and physical activity as obtained from the 34 participants at month 14

Drop-outs during trial: At the end of the recruitment, 96 registrations were received, of which 33 were excluded due to the selection criteria. From the 63 included remaining participants, 10 withdraw due to not having the time to enter the study and considered the project as to complex due to the used technology. The trial started in mid-February 2020 with 53 participants and concluded in the end of April 2021. Further drop-outs (n=19) were reported until month 9, the remaining participants stayed in the trial until month 14, resulting in a dropout rate of 36%. However, due to COVID-19, personal meetings were only possible for a limited number of cases (n=10), to that complete datasets with also blood pressure etc. were only obtained from 10 persons (Figure 4). The majority of participants were elder females, being married, often with a diploma of secondary studies, and most had an income of 750–1499 Euros per month (Table 5).

Table 5 Overview of socio-demographic aspects of participants at onset of study in Portugal

Statistical approach for both countries

Normality of data distribution and equality of variances were verified by Q-Q plots and boxplots, respectively. Log-transformation was carried out if data was non-normally distributed. To investigate changes over time of primary outcomes, for each country, linear mixed models were created, with outcome (BMI, blood pressure, questions related to acceptance etc.) as the dependent/observed variable, and timepoint (0, 3 months for the NL, 0, 9 and 14 months for Pt), gender (man, woman), subject ID nested within gender, and interaction of timepoint × gender to see if men and women differed regarding outcome changes. Age was inserted as a covariate. A p-value below 0.05 (2-sided) was considered statistically significant. All analyses were carried out with SPSS vs. 25 (IBM, Chicago, IL). Given Fisher-significant F-values for timepoint, least significant difference (LSD) post-hoc tests for all group-wise comparisons were carried out.

Ethical aspects

The study in the Netherland was exempted from Ethical Review Board scrutiny as per written decision letter of the Central Committee on Research involving Human Subjects (CCMO, The Hague, the Netherlands) according to Dutch law. The study in Portugal was approved by the Ethical Committee of the SCMP. Written informed consent was obtained from all subjects and the risks and benefits of participating in the study outlined in lay language. The GDPR guidelines regarding conducting human studies were followed.

Results

Dietary app and number of meals created

The dietary app was developed and further improved during focus group meetings and the ensuing field trials. For usage of the app to create meal plans see individual trials. A drop in participation rate as shown by the number of meals generated was apparent in the Netherlands, but not in Portugal (Figure 5). While in the Netherlands, up to 20–40 meals were created during early phases of the trial, this dropped to below 10 meals per day in the later phase. In Portugal, the initial number of meals created increased from 20–30 to over 60 in the final phase of the trial.

Figure 5 Number of meals per day planned in the Netherlands across all 3 field trials (upper panel) and the field trial in Portugal (lower panel).

Focus groups

The 3 focus group meetings took place in order to overcome limitations of the app in its present form. Among the main limitations of the developed dietary app and eHealth solutions in general, a number of points were mentioned by participants that were considered main barriers for usage and thus uptake by larger populations (Table 6). These concerned mostly technology and nutrition related aspects.

Table 6 Major barriers for usage and uptake of the developed e-Health solution for the elderly as highlighted by the focus groups in the Netherlands and in Portugal

Field trials

The purpose of the field trials was to test the developed and in the participatory design improved app in mid-long term trials in two cultural diverging European set-ups with the elderly, to test adherence and measure potential changes in especially anthropometric endpoints. For this purpose, online usage was registered by Firebase Analytics, acceptance of the developed solution was determined by questionnaire, and weight was recorded during the field trials. Furthermore, additional qualitative feedback on the usability and the recipes was collected in both field trials.

The Netherlands

Online usage: In trial 3, 86% were online daily. The rest went online a few times per week (10%), weekly (3%) or less often (1%). The participants were on average 13.3 h per week online (2 h/d). Aggregated results from all 3 studies showed that in total, 531 meal plans were created, with an average of 6.6 meal plans per participant. A total of 871 food items were replaced (average 10.8 items per user).

Acceptance of LIFANA solution: When comparing the mean of the responses of the acceptance questionnaire at timepoint T=0 and after the end of the trials, there was a significant increase in the rating of all questions (P<0.01, Table 7). Out of all 6 questions posed to the participants, all were less favourably answered than at onset. The average change was from 2.56 to 3.57, i.e. equivalent from rather agree-neutral to neutral-disagree.

Table 7 Changes of answers over time regarding acceptance of the LIFANA solution in participants from the Netherlands in trial 3 (n=23 participants). Timepoint 1= onset of trial, timepoint 2= at the end of the trial (month 3). Participants were able to choose from the following answers: 1. Fully agree. 2. Agree. 3. Neither agree nor disagree. 4. Disagree. 5. Fully disagree. Values shown represent the average answer of participants

However, out of the 23 persons interviewed, 1 stayed neutral regarding how they answered the questions over time, 20 gave on average more negative answers over time, 2 gave on average more positive answers over time, i.e. higher agreement. Thus, in approx. 10% of the persons, the LIFANA solution was more favourably evaluated with time of usage. Regarding the question of the future use of LIFANA, one person agreed that he/she will use LIFANA in the future, 6 were neutral on this question, 16 disagreed with this statement.

Outcome BMI: BMI was measured in 32 participants (20 females, 12 males) in the trials 2 and 3 (combined) in the Netherlands. Trials were investigated combined for increased statistical power and due to similarity of design and intervention. Participants were either having normal weight (n=13), or were overweight (n=17), i.e. with a BMI of 25–29.9 kg/m2, and 2 had obesity (BMI>30 kg/m2). The net change of BMI over time was −0.32 kg/m2, which can be considered small, and was not statistically significant (P=0.101). ID (person participating) within gender and gender had a significant effect on the outcome (P<0.01), signifying differences in weight status between individuals and genders, respectively. The interaction of timepoint × gender was not significant (P=0.070), i.e. no gender significantly changed BMI during the trial period.

Participants socio-economic background: Comparing data from the Netherlands to Eurostat from 2020 (https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=ilc_di04), the average annual income of a person in the Netherlands was 25.821 Euros per annum (or for those over 65 years 20.604 Euros), compared with 36.000 Euros of the 23 participants who finalized the study. Thus, the participants came rather from a well-situated background, also considering that they were no longer actively working. Regarding secondary education, it was stated that 81% of adults aged 25–64 in the Netherlands had completed upper secondary education according to the OECD Better Life Index [35]. In the present study, 52% of participants reported a tertiary education (e.g. University degree), also in line with participants being well educated (Table 2).

Portugal

Online usage: A total of 720 meal plans were created in Portugal during the entire 14-month trial time. An average of 13.6 meal plans was created per participant, one meal plan consisting of up to 6 meals. We observed that participants started to create plans more frequently towards the end of the trials when the integration with Continente online was put into effect. In this respect, during a test phase lasting from January to April 2021, 713 items were added to the shopping list, 120 products were removed, and 198 exported to the retailer (Continente).

A total of 6790 items were replaced (an average of 128 per user during the 14 months of the field trials). When looking at the distribution of planned meal types, we saw that most people used LIFANA to plan breakfast, lunch, and dinner, and we could also see that almost no one planned night snacks. Recipes were rated a total of 1426 times, with an average of 26.9 per user. Recipes were rated with an average value of 2.5 (on a scale from 1 to 5). A total of 117 food restrictions were created by participants.

Acceptance of the LIFANA solution: The questions were answered by 15 persons (until month 14), or by 33 persons (until month 9). Regarding the evaluation with 15 persons, no significant influence of time on the scoring of any of the questions was noted. More precisely, regarding the individual questions, number 5 and 8 (data not shown) stayed rather unchanged, no question improved regarding the rating, number 6’s average slightly decreased, showing a lesser agreement. Regarding the individual participants, out of 15, 6 stayed neutral regarding how they answered the questions over time, 7 gave on average more negative answers over increasing time, 2 gave on average more positive answers over increasing time, i.e. higher agreement, 7 persons agreed or strongly agreed (1/7) that they would consider buying LIFANA if it were on the market.

Upon re-analysis of the data with 33 participants, comparing timepoints T=0 months vs. T=9 months, similar results were obtained, i.e. no significant changes of any of the questions over time, except for number 3 (Table 3), where a slightly and significant higher, i.e. less favourably score was noted. Regarding the evaluation of the questions, out of 8 questions, ratings for two questions improved over time (higher agreement), i.e. number 4 and 8. Six decreased over time (lower agreement). Out of 33 persons answering the questions, 13 stayed neutral regarding how they answered the questions over time, 13 gave on average more negative answers over time, i.e. disagreement, 7 gave on average more positive answers over time, i.e. higher agreement, 10 agreed or strongly agreed (2/10) that they would consider buying the LIFANA solution if it were on the market.

Outcome BMI: The effect of time over 14 months on BMI of the 34 participants was not significant (P>0.05) and changed from 26.56 kg/m2 to 26.79 kg/m2; as was timepoint × gender interactions. Gender had a significant effect, as had the ID within gender (both P<0.001), again indicating significant gender and interpersonal differences of weight, respectively.

Other health-related outcomes: Considering data from all 3 time-points over the 14 months trial (n=15), there were no significant changes of blood pressure over time (P>0.05). However, for systolic blood pressure, a significant gender interaction with time existed (P=0.001). When re-analyzing data per gender individually, it was observed that the systolic blood pressure for men increased with time (P=0.011). No significant changes were registered over the 14 months of time (n=15) regarding waist-to-hip ratio. Likewise, no significant interaction of gender and time was found. Regarding percentage of fat mass, though the interaction of time and gender was not significant over the 14 months of time, a significant effect of time on fat mass was registered (n=10, P=0.008), i.e. a decrease from 31.8 to 26.9%. A tendency (P=0.058) for an increase in the percentage of percentage body water was also observed in these persons with time. However, as changes between month 9 and month 14 were more pronounced (compared to changes from month 0 to 9), and these were the months from November to April, seasonal reasons as an explanation cannot be ruled out.

COVID-19 related questions: A slight decreased physical activity level was reported by the majority of people (Table 4), together with a rather increased amount of food consumed. However, the majority of participants did not report altered cooking behavior or changes regarding the type/quality of food items consumed. COVID-19 also appeared to increase the perceived level of stress, with about half the participants reporting no changes, the other half reporting slight increased levels of stress. Despite the possibly increased amount of food intake and the reduced physical activity level reported by the participants, this did not seem to impact BMI or other anthropometric measures as outlined above.

Participants’ socio-economic background: Comparing this data (Table 5) to Eurostat from Portugal (https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=ilc_di04), the average annual income of a person in Portugal in 2020 was 10.800 Euros (or 9554 Euros (2019) per year for those >65 years), compared to 37844 in Luxembourg, which was ranked highest for European countries. Thus, the participants with an average income for most participants between 750–1499 Euros per month) came to a large extent from a somewhat average income category, also considering that they were no longer actively working. Regarding secondary education, it was stated that 48% of adults aged 25–64 in Portugal had completed upper secondary education (http://www.oecdbetterlifeindex.org/countries/portugal/), and also in the present study the most populated category was that of secondary education/diploma. A tertiary education was stated to be achieved in Portugal by approx. 37%, as compared to the NL with 52%. In short, the majority of persons participating in the study had a somewhat average if not slightly lower than average education in Portugal.

Qualitative feedback

Besides the measurements and specific questionnaires, we also collected qualitative feedback from the participants before and after the trials in form of additional questions asked. In The Netherlands, participants were asked at timepoint T=0 “On what would you like to receive support when it comes to nutrition and exercise?”. Half of the participants mentioned “maintaining a healthy weight” and “control calorie intake” (52%), followed by “receiving advise and tips on health” (41%), “information on healthy nutrition in general” (40%), “information on how many calories are burned when doing certain exercises” (38%), “how to compose healthy meals daily” (36%), “how to compose healthy meals weekly” (33%), “get insight on what I am eating” (28%), “suggestions for snacks that are low in calories” (26%), and “short videos in which a recipe is explained and prepared” (21%). LIFANA covered most aspects except for video instructions. Insights on meals were provided in the dashboard as weekly summary of nutrients, but without additional explanation or discussion.

After or toward the end of the trials (in NL 3rd trial timepoint T=3 months and in PT T=8 months), participants were asked to comment on positive and negative points they experienced. The feedback could be roughly classified into the following categories: general usability of the application, acceptance of the recipes and meals, and quality of the meal plans. In the following, we summarize and discuss the main points. The full list of comments can be found in electronic supplementary material (ESM) 1, Table 1.

Usability as determined by direct feedback of users from focus groups and trials

Participants in the Netherlands found the installation of the app from the appstore difficult, because it required access to an invitation email with a link to download the app from Apple Testflight. Some participants lost their passwords during the trials and required assistance to reset it. In Portugal, this was less of an issue since the participants received frequent visits from their caregivers that supported the trial. For future projects, it would be advisable to send PIN codes by mail to avoid password problems. Besides that, participants liked the design, readable fonts, photos of the meals, and easy navigation within the app.

Meal plans and recipes – feedback from users from focus groups and trials

Users in The Netherlands found it useful to configure food restrictions by adding foods to a blacklist. However, they criticised that the way it was implemented (by selecting all foods individually from the Food Composition Database) takes too much time. In some cases, foods still appeared in the recommendations. We addressed this issue in the beta prototype and provided predefined restriction profiles for certain diets, e.g. vegetarian diet. Participants in the Netherlands liked the variety of the recipes and how the plans helped them to structure their meals and avoid snacking. Users in both countries criticised cases of repetitions of the same meal for lunch and dinner. They also found some suggestions not appropriate to their habits and food culture, despite the local recipe datasets. In the Netherlands, seniors reported to consume mostly bread for lunch and more ‘heavy’ cooked meals for dinner. In LIFANA, the meal planner did not implement their preference for a light lunch. Likewise, participants in Portugal found recipe recommendations for lunch too elaborate on workdays. In general, users complained that grocery shopping was time consuming, and the system did not consider leftovers, causing food waste. While some participants desired simple and convenient meals, others complained that the recommendations were not really healthy, since some recipes were based on processed food products instead of fresh ingredients. We conclude that different types of recipes, with different types of processed ingredients, are required to fully satisfy both groups.

Discussion

We have reported the iterative development – based on focus groups – of a novel meal recommender plan especially suitable for the elderly, oriented toward their caloric and protein needs and proposing balanced meal suggestions, as well as its successful application in field trials with participants living in two different cultural European settings, i.e. the Netherlands and Portugal. The main findings were that during the intervention with the LIFANA solution, participants did not change weight or other anthropometric and biological markers related to health, despite the interference with the unforeseen COVID-19 crisis that had been discussed to be related to potential weight gain [36]. However, improvements were also not observed though were at least regarding weight, also not targeted. Regarding the acceptance of the developed solution, while it was reasonably well evaluated in Portugal, participants in the Netherlands were more critical.

Despite the apparent higher motivation to use and accept the developed solution in Portugal compared to the Netherland, as shown by the lower dropout rate and more optimistic ratings of the LIFANA solution, the field trials with the dietary recommender app did not result in any significant changes in body weight or other anthropometric measures in both countries, or in Portugal, blood pressure and body composition, except for small changes in %age fat mass and %age body water, which may have been attributable to seasonal fluctuations. Former studies with dietary apps have reported in part slightly more pronounced results, e.g. with reductions in BMI of 0.3 kg/m2 for 1–12-month interventions, though these were adults with chronic diseases (mostly obesity) [37]. As in our study, elderly were rather of normal weight or overweight, weight stabilization was regarded already as a positive finding, especially in sight of recommendations that normal or overweight elderly are not recommended to lose weight in order to reduce the risk of sarcopenia [38]. Furthermore, as during the COVID-19 crisis, some reports about weight gain in adults were reported in several [39, 40], though not all populations [41], stabilization of weight was already regarded as a positive outcome. Weight stabilization was found in the present study, despite the subjectively reported decrease in physical activity in Portugal and the more stress-related eating behaviour according to the COVID-19 questionnaire, behaviour that has been reported to trigger more uncontrolled eating behaviour and weight gain [42].

The somewhat limited adherence to the developed solution is reflected in the answers to the LIFANA solution satisfaction questionnaire. In the Netherlands, participants, after having used the solution for >3 months, were less optimistic that the solution would help them to improve eating behaviour, better food planning, improving weight, and also not regarding having no problems using it, knowledge improvement and the likelihood in using it after the end of the trial. This was different in Portugal, where similar questions did not result in less optimistic/affirmative answers compared to onset, except for the improved planning of personal finances. Likely, these findings could be explained by the more closely following-up of participants in (except for interruptions during COVID-lockdowns) weekly personal visits by the care givers from Santa Casa in Portugal, as well as the somewhat lower socio-economic background of participants in Portugal, making them more likely to appreciate any potential help, even though appreciation has in other studies not been regarded as vital for study participation [43]. In addition, the link to the online shopping app in Portugal may have nudged persons into a more continuous usage, as observed by increased frequency of use following its inception. The more personal follow-up by personnel from Santa Casa in Portugal likely also explains the lower dropout rate of 36% as opposed to 68% in the Netherlands. Indeed, studies have pointed out the importance of personal meetings, such as with nurses or other trained personnel, to reduce dropouts in eHealth interventions [44]. Some eHealth studies have set a 50% dropout-rate as a threshold above which significant bias is introduced to the evaluation [45]. However, also classical studies with multiple counselling sessions have been reported to suffer from dropouts around typically 35% [46], making well-designed eHealth interventions a viable alternative or an add-on. The main learning lesson to reduce dropout rate is likely the closely and personal follow-up of participants.

Somewhat surprisingly, regarding user participation in this present study, more meal plans were created on average per participant and unit time in the Netherlands than in Portugal (10.8 during 3 months vs. 13.6 during 14 months). When comparing the number of food items replaced (10.8 per user in the Netherlands during 3 months, vs. 128 items per user over 14 months), slightly more items per unit time were replaced in Portugal, perhaps suggesting a more thorough interaction with the LIFANA solution, though this remains speculative. However, the number of meals planned and food items replaced decreased considerably form the onset of the trials to the end in the Netherlands, while it increased in Portugal, probably due to the availability of the Continente online shopping function that was added toward the end of the intervention – otherwise, there was no significant decrease. Pronounced declines in participation rates and user times in mobile app devices targeting life-style interventions over the study period have been emphasized as a problem earlier [47], in part due to also technical issues [48]. Thus, ease of use and features engaging the user were recognized as vital tools to increase adherence in the present study also.

Therefore, the main obstacles that were perceived for a successful uptake of the LIFANA solution included adapting the meal recommender to the local food context, keeping participants motivated in the long term, and user-friendliness (usability) of the solution. The local socio-demographic and foodscape context between the Netherlands and Portugal was very different, with participants in the Netherlands having a higher education, a higher income, and apparently a more critical view on dietary eHealth interventions and their benefits. Dietary habits also appeared to differ largely. For example, it was stated by participants in the Netherlands that heavy meals for lunch are uncommon, with a preference rather for bread-based lunches, and that the suggested meals were perceived as being “too heavy”, i.e. caloric, or took too much time to prepare. This is corroborated by studies reporting that in the Netherlands, bread was the preferred energy source for the Dutch adults for lunch [49]. This differed from Portugal, where for lunch, often warm meals including e.g. soups were consumed according to participants. Also the energy intake distribution over the day could differ between countries, however, according to reports encompassing results from the Netherlands and Portugal, the energy intake appears rather evenly distributed over the day in both countries, with no clear peak intake in the first half or second half [49, 50, 51]. In addition to the time and relative weight of the various meals, local adaptations were needed to incorporate typical local dishes. In this sense, for the Netherlands, a new meal database containing ca. 300+ recipes selected from the Dutch Voedingscentrum was created, while the Portuguese database included ca. 400+ recipes created by nutritionists in the previous projects SousChef [25] and CordonGris [28]. All recipes were rather unique for their local background. These findings highlight that is crucial to assess typical food habits prior to the application of dietary suggestions, considering in detail person’s needs and traditional dietary patterns of the region. While participants of the focus groups in the Netherlands were initially asked to give examples of typical meals, the sample size was too small for a systematic analysis of local food habits and culture. Therefore, a local adaptation to the food culture and also the ability of the software to adapt to persons needs with time, such as be intelligent, machine-learning recommender systems [52] are considered paramount.

The observed decrease of use and motivation during the trials in The Netherlands indicated a need for specific motivational features, which were lacking in the app. Indeed, such features, including gamification effects [53], nudges such as reminders [54] or further explanations [55] and virtual prices to some degree [56] have been emphasized as tools to increase adherence to e-Health solutions. The declining adherence was also predicted by some participants of the focus groups based on their prior experiences. While such motivational features were discussed in the requirements definition phase by the partners, priority was given to improvements of the core functionality, i.e., the meal planning algorithm and integration of regional recipe content. A recent systematic review and meta-analysis [8] has judged the effectiveness of app-based mobile interventions in the area of nutrition to be promising, but highlighted main limitations, including a) the need for time-consuming food logging/monitoring; b) being unsuitable for some populations, e.g. the elderly, less familiar with apps; c) focus on interventions only via diet or PA (mainly moderate-to-vigorous PA); d) overlook context (cultural, socio-economic, environment); e) fail to engage on the long-term via e.g. gamification/nudging features; f) fail to integrate psychological aspects, e.g. affective states. While we strived to address some of these issues in the present project, the way forward would point toward including motivational features or nudges and psychological aspects into account also, which has already been highlighted as a main barrier in previous studies [57, 58].

The study has several strengths and limitations. A main limitation of the study was its longitudinal design without a control group, which does preclude inferring causal relations, and may be prone to changes over time such as seasonal aspects or unexpected events, such as the COVID-crisis. Indeed, another obstacle was the interference of the COVID-19 crises and lockdowns that limited taking more objective measures consistently, such as BMI, rather than relying in part on self-reported measures. Biological samples, e.g. with respect to blood lipids, oxidative stress or inflammation, as more objective markers were not collected to lower invasiveness of the study and to foster a higher participation rate, but would have allowed for studying more subtle metabolic changes over time. In addition, a complementary food frequency questionnaire would have given more in-depth details on the dietary patterns. Furthermore, as also highlighted in previous reviews [15, 16], improving learning algorithms that could adapt better to individual preferences would likely improve further adherence to a developed meal recommender, as may be an app that shows more visual information of ingredients/meals, though the latter is time consuming to develop. Strengths included the reliance of the recommendations on scientific-based dietary targets for calories and proteins for the elderly, iterative development of the LIFANA solution with stakeholders including primary users and professional and family care-takers, conducting the trials in different socio-economic and cultural backgrounds, the tailored design for elderly encompassing aspects of their local cuisine and dietary preferences including allergies that could be entered into the mobile app, as well as the reasonable long-term of the interventions, up to 14 months, and choosing two rather different cultural backgrounds in Europe. Regarding the meal recommender app, one key aspect was the precise information about the intake of nutrients and energy in mixed dishes or multi-ingredient foods, which represent the majority of items in diets worldwide. The meal planning approach had the advantage over food logging methods, such as food frequency questionnaires, that the recipe database used provided information about ingredients and cooking methods.

Conclusions and main challenges encountered

We developed a personalized meal recommender mobile application for the elderly that included dietary preferences, age, gender, physical activity and budgetary considerations and targeted a balanced diet oriented around a caloric goal and protein intake. This was iteratively improved by focus groups and then in field trials. Regarding usability, it was found that the frequency of employment of the LIFANA solution by participants was improved by additional features, namely the ability to produce a shopping list and considering budgetary aspects. Acceptance over time also dropped if the solution was not backed up by personal visits with caretakers. Using the application and the meal recommender system over several months in field trials did neither improve nor resulted in deterioration of participants weight status or other markers of health assessed, which on one hand may be attributed by the somewhat limited use by participants, but on the hand weight loss was not a goal of this study in the elderly, and results may somewhat been distorted by the parallel COVID-crisis. The main perceived challenges along the way toward developing a meal recommender system for the elderly were:

  1. 1.
    High drop-out during the long-term trials, especially when not followed-up by health professionals and in the absence of additional motivations such as budgetary considerations;
  2. 2.
    Limited integration of local recipes and insufficient knowledge of the recommender system regarding local meal culture or inability to rapidly adjust to user suggestions, such as types of foods typically consumed for lunch vs. dinner, which could strongly differ between cultures;
  3. 3.
    Limited user-friendliness of the app, time-consuming features, and outstanding features compared to other products on the market.

Therefore, major obstacles to overcome toward higher acceptance of the eHealth solution would likely include additional motivational and engaging features, possibly integrating psychological/behavioural aspects, in-depth recognition of the local food-culture, and assuring personal contact with caregivers via the app, at least intermittently. Finally, a vital consideration to any validation of the app should be a field study demonstrating its usefulness, preferable a study of randomized design with a parallel control group, as unexpected changes to the environment, such as COVID, may happen longitudinally. In summary, we thus propose especially further integration of motivational features and psychological aspects as the next level to further improve adherence to such meal recommender solutions.

The authors thank all the participants who took place in the field trials. The authors also thank all members of the LIFANA project consortium for their support of the study, these include the end-user organisation partners, i.e. Nora Ramadani (KBO-PCOB, NL), Sandra Arouca, Berta Brito, and Silvia Jesus (Santa Casa da Misericórdia do Porto, NL), our technology partner, i.e. Katja Verbeek (Gociety Solutions BV., NL), and research partner, i.e. Jorge Ribeiro and David Ribeiro (Fraunhofer, PT).

References