Developing a Web Credibility Evaluation Tool Using PROMETHEE Method

In recent years, the Internet has become an indispensable way for users to find information which is almost instantaneously available. However, the presence of information on different websites makes the user needs to precheck the credibility of the selected websites. Most users find it difficult to assess website credibility in terms of its particular characteristics or factors. Accordingly, we proposed an automated evaluation tool which considers various factors to assess the credibility of different websites and rank them from the highest credibility score to the lowest in order to allow the user to select the most credible website. We used the Preference Ranking Organization Method for Enrichment Evaluations (PROMOTHEE). The latter is one of the Multi-Criteria Decision Making methods (MCDM). It combines pairwise comparison and outranking methods in order to give more accurate and superior credibility scores due to its enrichment evaluations. For the proposed tool to be acceptable, we carried out a correlation analysis to determine the coefficient of correlation between human judges and the proposed tool. We found the coefficient of correlation rho is 0.943 which indicates that there is a strong correlation between the human judges’ ranking and the ranking given by the proposed website evaluation tool.


Introduction
Today's generation relies on the web to allow access to vast amounts of information.The existence of many websites that provide knowledge about seemingly limitless topics makes an individual's life more productive.Due to the open nature of the web, anybody can own a website and distribute information; the problem arises in that such websites lack any verification of their credibility.This can have an enormous impact on individuals' everyday activities due to their dependency on the web to make certain decisions.It is essential for a client to be able to authenticate a website as being credible and safe to use (Aggarwal & Oostendorp, 2011).In this case, credibility implies the certainty a user can presume from a given website.Fogg et al. (2002) described credibility's influence through prominence and interpretation.This means that users might notice something in particular on a website and make specific judgments about it.Credibility indicates the objectivity and subjectivity components of the believability of sources.Credible webpages should possess a high level of trustworthiness and demonstrate proper expertise.These two factors include the criteria that the website is professionally designed and logically arranged.Besides these, it should be kept as current as possible and secure.Users typically lack knowledge about the factors that can affect website credibility.Accordingly, it is important to support users with the proper tools to assist them in evaluating website credibility to prevent them from being influenced by misleading information.Thereupon, users become more confident about the websites they are visiting because they were previously unable to differentiate if these websites were credible or not.Additionally, they will be able to achieve a better understanding of the content.Using such a tool enhances users' perception of credibility and makes them more knowledgeable (Walraven, Brand-Gruwel, & Boshuizen, 2013).In this study, we developed an automated evaluation tool that uses the PROMETHEE method and incorporates various factors to give more accurate and superior credibility scores.The proposed tool will rank the chosen websites from the highest credibility score to the lowest.We hypothesize that our proposed tool can give valid credibility scores of websites and results in a strong correlation between the human judges ranking and ranking assigned by our proposed tool.This paper is organized into seven sections: Section 2 presents a thorough literature review of web credibility factors and tools for automation of credibility evaluation.Section 3 gives the methodology for designing the proposed tool.Section 4 shows the case study for evaluating a set of health websites.Section 5 presents an empirical evaluation to check the tool's validity.Section 6 presents the main research results of the study.Section 7 gives the conclusion and foreseeable future work.Lowry, Wilson, & Haig (2014) have demonstrated that webpage presentation and layout can trigger more positive judgments of credibility.Similarly, Fogg et al. (2003) conducted quantitative research by asking 2684 participants to evaluate the credibility of two web pages.They obtained the participants' comments in order to discover features that most gained users' attention.They concluded that website presentation and layout was commented on most frequently at 46.1% as seen in figure 1.In contrast, the most negative factor noted in these evaluations of web credibility was affiliations.Sundar (2008) proposed the MAIN model to help users evaluate websites.This model involves 26 factors that influence judgment on credibility, it depends on a set of studies that encompass a group of web credibility factors which are shown in figure 2. According to Lucassen, Muilwijk, Noordzij, & Schraagen (2013) proposal in the 3s-model, there are three strategies of web evaluation that must be in place.The first is the consideration of the semantic features which comprise the preciseness of the website.The second is the features found on the surface of the website, which includes the design, text color and web aesthetics.The third strategy is past experience with the information being used and its sources which are used to describe the affiliation of the website owner.Westerman, Spence, & Heide (2014) focused more on recency or updates.They claimed that periodic updates on a website drive increased credibility.Shneiderman (2015) concentrated on how to evaluate website content and what has to be done in order to increase its credibility as well as the reliability of the resources.Teng, Khong, Goh, & Chong (2014) asserted that online reviews provided by specific users on particular websites affect others' decisions to proceed with their use of the same website.In addition, they presented the main aspects of website quality, namely accuracy and relevance.

Web Credibility Factors
Further, Diana, Bahry, Masrom, & Masrek (2017) reported a list of credibility types.In detail, these types included surface, sources, content and reputed credibility.To demonstrate, surface credibility refers to the extent to which the site presentation is considered reliable by its users.In other words, Wathen & Burkell (2002) described this type as a straightforward assessment of superficial characteristics.Another study presented by Kim & Fesenmaier (2008) asserted that site appearance is the first factor that impacts on site users' judgment.Various factors of this type have been determined which are aesthetics, usability, content and navigation.Robins & Holmes (2008) claimed that the typical user takes just 3.2 seconds to make a judgment on website credibility, as based purely on its surface.Webpage design components include images, colors, font, easy to use menus and so on (De Angeli, Sutcliffe, & Hartmann, 2006).Source credibility which is defined clearly by Hu & Shyam Sundar (2010), refers to the extent to which beneficiaries see the credibility of the information sources.In other words, source credibility is likewise prescribed to be the user's expectation of the credibility of a source.Content credibility refers to the messages themselves; for instance, information precision (Metzger, Flanagin, & Zwarun, 2003).Olteanu, Peshterliev, Liu, & Aberer (2013) stated that this refers to the reliability of content factors which are textual content and website design.Hasan (2013) has provided a number of indicators related to content credibility by which to evaluate the quality of a webpage such as accuracy, objective and aesthetic.Finally, reputed credibility is described as the extent to which the object's reputation is perceived.This is due to what is revealed about it to a third party.To put it differently, this indicates the ranking of a website by some specific third party (Christiansen, 2013).This type can be obtained based on prestigious awards, authors' expertise and so on (Liu, 2004).Aggarwal, Van Oostendorp, Reddy, & Indurkhya (2014) focused on specific factors when they implemented their system regards to web evaluation.These factors are webpage type, sentiments outcome, google page ranking, latest update, reputation and reviews.The latter factors they implemented do not prevent the authors from noting the other factors that they obtained from a thorough study they conducted as shown in table 1.

Factor Definition
Type of website This is evident by the URL used for that particular website.Websites that are used for non-commercial purposes tend to be more credible.

Website update
A website whose information is regularly updated is considered to be more credible than one that is rarely updated.

Web document Source
A primary document means there is no possibility of manipulation unlike a secondary one.Therefore, a primary web document is more credible than the secondary one.

Contact information
Availability of the website owners' contact details which include email address etc. through the provision of contact information on a website, the owners can be contacted if any clarification is needed, thus making them more credible.

Header and footer
This is done mainly when assessing the trust of the site.The header should include the enterprise logo while the footer should encompass the information pertaining to the

Accuracy, unambiguity and completeness
These three points increase credibility because there will be no scope for interpretation of the information that is shown in the web document.

The sentiment analysis
This will help in deciding if the owner's opinions are biased or not.

Webpage design
It increases the level of credibility.Its graphics and texts ought to be proper.
The tone of the webpage It includes ironic, humorous or exaggerated.One such example is a site with exaggerated controversy with unassociated proofs.

Quality of content
This can be divided into three types: elementary, technical and advanced.In any case, a high-quality webpage is considered to be more credible.

Google page rank
It is used to specify the popularity of a specific website among its users.

Advertisements
Check if a similar individual or association is providing the advertisement and the information content.Assuming this is the case, advertisements are considered to be of biased informational content.Generally, advertising diverts users' attention.Therefore, it can either reduce or add to credibility.

Software requirements
Specifying software requirements that constrain access to a website.

Automatic Web Credibility Evaluation Tools
There are a number of studies that have utilized weighting schemes due to combining various factors and predicting credibility scores.For example, Yamamoto & Tanaka (2011) proposed a framework that enables clients to judge the credibility of search outcomes and obtain credible web pages.The framework calculates and visualizes different engine outcomes scores by taking significant credibility principles into consideration.In addition, it predicts the user's credibility judgments and modeling user comments about the engine outcomes.Therefore, it reranks the web engine outcomes depending on users' predicted credibility model.The researchers infer that their system is more valuable to the user as they can effectively obtain more credible web pages than through traditional web engines.Aggarwal & Oostendorp (2011)  Some studies have presented solutions that depend on machine learning approaches.For example, Sondhi, Vydiswaran, & Zhai (2012) explored the possibility of automating the process that assesses the credibility of medical web pages.The motivation behind this study is the lack of automatic tools which classifying medical web pages as being credible or not.Of course, manual evaluation is not able to handle an extensive number of medical web pages.In order to automatically predict the credibility of the study domain, the authors proposed a supervised learning approach.After they completed experiments on their dataset, they achieved an overall prediction accuracy of 80%.Also, Olteanu & Peshterliev & Liu & Aberer (2013) automatically assessed the credibility of web pages by utilizing machine learning algorithms without paying specific attention to any particular sort of web content, which made their approach more generic.They identified a list of features related to web credibility assessments.Next, they studied web page characteristics by exploring different data sources with the end goal of building a generic-purpose automatic web credibility tool and to enhance the accuracies of machine learning algorithms.Finally, they performed various experiments on a real dataset which yielded good outcomes.Their approach achieved an accuracy of 75% in terms of classification.However, it recorded 53% for regression.

Design
We mentioned different factors that should be considered for credibility evaluation above.In this study, we automated credibility assessments using 13 factors that are considered important to rank websites depending on their credibility scores.These factors include usability, completeness, software requirements, link integrity, date of last update, reputation, website design, accessibility, sentimental value, website tone, quality, page rank and security.Furthermore, the tool relies on a number of application programming interfaces and website tools to find the specific values of the selected factors.For each factor, the matching API is listed below in table 2. It is important to note that all chosen factors are readily available through the APIs and therefore our website evaluation tool relies on automatic website scraping.It is also important to note that some factors such as website usability and accessibility are aggregations of various supporting factors.For instance, website usability is a combination of website responsiveness (ability to adjust to various types of computing devices i.e. mobile phones, desktops and other devices of diverse sizes) and loading speed.Website accessibility is a combination of website availability and ease of access to information on various platforms.Figure 3 displays the proposed automated credibility evaluation process.The user enters a list of URLs to be evaluated for credibility.Once the URLs have been submitted, the web scrapping subsystem of the website ranking tool initiates the extraction process of factors' values for each website in a sequential manner as per the listing of the websites in the input pages.After all the factors' values have been extracted, the preprocessing process prepares them for the PROMETHEE method which involves cleaning and sorting.To compute credibility scores for all the websites depending on the selected factors and their weights, we used the Preference Ranking Organization Method for Enrichment Evaluations (PROMOTHEE).The latter is one of the Multi-Criteria Decision Making methods (MCDM).This method has several advantages such as its ease of use, that it is best suited for quantitative factors, its ability to make enrichment evaluations because it combines pairwise comparison and outranking methods, it does not require initial assumptions, it is elastic and can accommodate a large number of factors (Hester & Velasquez, 2013).

Implementation
WebCredible has been implemented using the Java programming language because it is a secure programming language with robust packages.It has a wide array of inbuilt libraries and programmers have the leeway to add additional customized libraries.This is particularly advantageous since a developer can extend the language library to package additional functionality into the language.It is widely used for web applications and services due to its rich web application libraries.It has Java servlets and the ability to package data in JSON and other scripting languages to facilitate seamless communication between a client and server (Sebastian, 2010).The tool will be implemented primarily in Java but will utilize HTML, servlets and PHP for the purposes of extracting the required factors' values and displaying the results of the PROMETHEE method.Selenium is a process for automation of browser reading.It reads the content of a browser as a human being would do.This is an ideal application for this project since it enables the system to extract the values of the factors required from existing APIs in an automated manner.It supplies the APIs with the URLs of the websites that the user desires to evaluate and rank.The APIs will return the desired factors' values which will then be captured automatically by Selenium.The factors' values will then be passed into an array in readiness for further preparation and processing.ChromeDriver is a chrome add-in that works with Selenium to launch and manage the chrome browser automatically in order to make Selenium do its expected tasks.ChromeDriver manages the browser sessions through the opening and closing the browser (Taracha, 2017).Throughout the whole web scrapping process, the browser will keep opening and closing as dictated by the ChromeDriver.The activities of the automatic browser will be not visible to the system user.
The proposed website evaluation tool will rely on various libraries for the implementation of the PROMETHEE method.Since the computation of the ranking will be done on the server-side, Apache Tomcat v9.0 will be used as the web server for the website ranking tool.This is mainly because this server comes bundled with the Eclipse IDE and integration is easy to understand and implement.The tool will be implemented by using Java Runtime Environment (JRE), Web App libraries and user-defined MCO Java libraries designed specifically for implementation of the PROMETHEE process.The stepwise procedure for implementation of the PROMETHEE method is outlined below.

• Determination of Deviations Based on Pairwise Comparison
This is where the weights are determined and numerical values are assigned.The pairwise comparison also takes place here for different criteria.Pairwise comparison is simply the numerical difference between two conflicting preferences (Behzadian, Kazemzadeh, Albadvi, & Aghdasi, 2010).

• Application of the Preference Function
There are various functions that can be applied at this stage to determine the degree/ range of difference between the two alternatives for each factor.In this case, we settled for a linear preference function that is simple, straightforward and easy to implement in a programming language.The linear preference function has two parameters, namely the indifference and preference thresholds.If the difference is less than the indifference threshold, this means that preference degree is equivalent to zero.However, the preference threshold will determine when to prefer one factor over the other (Mladineo, Jajac, & Rogulj, 2016).The preference threshold for this study will be set at 0.2 implying that when the relative difference is more than 20%, one solution will be preferred over the other.In case, the difference is between these parameters, the linear interpolation will be applied.
• Calculating the Global Preference Index This step will take place after all pairs of alternatives have had their preference degrees for all factors.It will be applied to globally compare each pair of alternatives.Along these lines, a complete outranking relation is built (Mladineo et al., 2016).

• Calculating Outranking Flows
To position each alternative with respect to others, the positive and negative flows are calculated for all alternatives.This step produces how an alternative performs better than all other alternatives which represents the positive flow.The same goes when the alternative is fairly badly against the entire set of alternatives.This is referred to as the negative outranking flow.This stage involves preferences, indifference and incomparability (Mladineo et al., 2016).

• Calculating Net Outranking flows
It is performed by calculating the difference between the two outranking flows for each alternative.This produces a ranked list for all the alternatives.In this stage, the entire set of alternatives are comparable.It involves preferences and indifferences (Mladineo et al., 2016).The output will be a value between 0 and 1 but the proposed tool will further convert it to a percentage in order to increase user understanding of the results.These steps are basically those of accepted mathematical functions and which are shown in figure 4 ( Behzadian et al., 2010).

Case Study: Health Websites Ranking
Health is an important field in the current world.The need for credible health information is on the rise as the world population seeks to embrace healthy living (Sbaffi & Rowley, 2017).The internet has become one of the main sources of information with regards to the health niche.It is therefore only prudent that the researchers sought to test the credibility of health websites.Table 4 shows the raw website list which was supplied to the tool.Comma separated values are listed contiguously and then separated by commas just as the name suggests.The input section of the PROMETHEE tool is shown in figure 5 with the comma-separated URLs copied therein.
Once the URLs have been submitted by pressing the "Submit" button, the web scrapping subsystem of the website ranking tool begins the extraction process of each factor for each website in a sequential manner, namely as per the order in which the websites have been listed in the input page.The extracted values and weights are then passed to the main PROMETHEE methods for pairwise comparison and outranking methods.The tool will rank the chosen websites from the highest credibility score to the lowest as displayed in figure 6.
Figure 6.Credibility score for each website

Empirical Evaluation
The website ranking tool underwent the scientific evaluation to gauge its performance against human judges.The judges were selected randomly from recent graduates.25 judges in total were selected and were given the website URLs in the same order that they were fed into the website ranking tool.They were then requested to determine a unified ranking order for the websites.The website ranking factors' values were not availed to the human judges and they were not made aware of all the factors to consider.After their evaluation, the human judges returned the ordered ranking to the researcher.The data was used to statistically evaluate the level of agreement between the website ranking tool and the human judges.The most appropriate measure of evaluation is the Spearman's rank correlation coefficient also known as Spearman's rho.This coefficient is used to find the relationship or correlation in the ranks of variables.In this case, the correlation of interest is the relationship between the ranking of websites by the website ranking tool and the ranking determined by the human judges.Spearman's rho ranges from -1 to 1.
In case rho lies below zero, the two rankings have an inverse relationship; where it is greater than zero, the relationship is positive.The greater the magnitude of rho, the stronger the relationship between the rankings ( Anonymous, 2017).The evaluation procedure was completed automatically using IBM SPSS (Statistical Package Figure 5. Tool Inputs for case study for Social Sciences).This package is preferred over Excel due to its ability to automatically calculate Spearman's rho.The inputs are in the form of two ranked lists, namely the human judges and website ranking tool.

Research Results
We ran a scatter plot on the input data, the results of which are shown in figure 7. The scatter plot is a graphical tool that shows in a rough manner, the nature and direction of the relationship between two variables.From the scatter plot shown above, data is monotonically increasing.This indicates that the data meets the basic requirement for correlation analysis.The other requirement for correlation is that data should be ordinal which is also true in this case.From the correlation results shown in table 5, the coefficient of correlation rho is 0.943.It is evident that the results are statistically significant at a level of 0.01 since the calculated p-value is 0.005 which is less than the set value of 0.01.This indicates that there is a strong correlation between the human judges ranking and ranking by the PROMETHEE website tool.

Conclusion and Future Work
Recently, the web has begun to play a major role in disseminating significant and shared information.Users are able to acquire and distribute information on the web.However, the information published lacks any pre-checking process.In this way, web data is not constantly exact or correct.Despite the importance of the web evaluation field, there were only a couple of pieces of research that have introduced practical tools with which to evaluate website credibility.However, some research has provided a set of measures by which to assess website credibility.Despite the existence of ready factors, users find it difficult to properly evaluate website credibility.Thus, a number of Scatter Plot researchers have introduced web credibility evaluation tools that employ these measures to assist users in evaluating web credibility in an automated manner.In this paper, we proposed an automated web evaluation tool to evaluate the credibility of websites.This tool shows the credibility score for each of the chosen websites as well as an associated ranking from the highest credibility score to the lowest.This should help the user to make an optimal decision when choosing the most credible website for their particular needs.We evaluated our proposed tool by requesting the human judge to determine a unified ranking order for the list of websites provided.The same websites were subjected to the tool.A correlation analysis was then carried out to determine the correlation coefficient between the human judges and the proposed tool.We found correlation coefficient rho to be 0.943 which indicates that there is a strong correlation between the human judges ranking and ranking by the PROMETHEE web evaluation tool, supporting our hypothesis.We can conclude that our tool is serving its intended purpose.For the future development of the proposed tool, we plan to involve more credibility factors in order to get better evaluation results.Also, we will modify our tool in a way that enables the user to enter a search topic instead of a list of URLs.Further, we intend to employ experts to achieve research experiments instead of students judges as well as increasing the number of participants.Finally, we plan to apply different techniques to record users' feedback on our tool in order to enhance its performance.

Figure 1 .
Figure 1.Participants' comments on the credibility of two web pages

Figure
Figure 7. Scatter plot

Table 1
. Several factors affecting web credibility analysis sponsors of the website.
Aggarwal & Oostendorp & Reddy & Indurkhya (2014)d factors in order to compute the credibility scores of websites.To validate the tool, the authors gathered credibility scores for a group of sites that were evaluated by human judges.They found that the correlation is low (0.484) between human judges and the proposed tool scores.Of course, this was because of the limited number of factors that were incorporated.Aggarwal & Oostendorp & Reddy & Indurkhya (2014)developed WebCAST to evaluate web credibility in order to produce a summary indication of web credibility.They likewise utilized the PAPRIKA method as it assigns specific weights to factors which represent the significance of the factors.They incorporated six factors which are webpage type, latest update, sentiments outcome, google pages' ranking, reputation and reviews.Moreover, they state that it is difficult to quantitatively measure the whole factors.Given that, they used only six factors which can be quantitatively measured.The same evaluation of the previous study was performed to determine a correlation 0.89 which is capable of generating credibility scores consistent with human judgments.
explored different factors that influence web-based documents.They presented an automated tool that assigns credibility scores to web documents depending on various factors and reports back to the clients.The proposed tool incorporates only four factors with which to assign a credibility score to a website.These factors are the webpage type, latest update, sentiments outcome and google pages' ranking.The authors used "Potentially All Pairwise Rankings of all possible Alternatives" (PAPRIKA).The latter is one of the Multi-Criteria Decision Making methods.

Table 3 .
Weights of the selected factors

Table 4 .
The chosen URLsThe PROMETHEE website ranking tool accepts input URLs as comma separated values in the large text box.

Table 5 .
Correlation results