Upravljanje kakovosti in čiščenje podatkov

PODOBNIKAR, UROŠ (author), Krisper, Marjan (mentor)

Današnje organizacije se pogosto soočajo z izzivom, kako obvladovati veliko količino podatkov, ki jih uporabljajo pri svojem poslovanju. Zaradi mnogih razlogov je zelo pomemben vidik obvladovanja podatkov tudi zagotavljanje in ohranjanje ustrezne kakovosti podatkov. V organizacijah namreč po eni strani ustrezno visok nivo kakovosti podatkov predstavlja konkurenčno prednost, po drugi strani pa slaba kakovost podatkov vodi v številne neljube posledice. V preteklosti so se izoblikovala ogrodja, metode ter orodja kot pomoč pri zagotavljanju ustrezne ravni kakovosti podatkov, poleg tega je kakovost podatkov obravnavana tudi v različnih standardih in zakonodaji. Kljub temu pa raziskave kažejo, da je stanje v organizacijah na tem področju še vedno razmeroma slabo. Namen naloge je raziskati in predstaviti področje kakovosti podatkov v organizacijah ter predstaviti problematiko, ki iz tega izhaja. Predstavljene so posledice slabe kakovosti podatkov ter vzroki, ki vodijo v takšno stanje. Podani so tudi razlogi, zakaj je kakovost podatkov v organizacijah pomembna, ter predstavljeni standardi in zakonodaja s tega področja. Problematika kakovosti podatkov se pojavlja tudi na področju interneta stvari, ki je v zadnjem času deležno velikih raziskovalnih prizadevanj, zato je obravnavano področje prikazano tudi iz tega zornega kota. V nalogi je največji poudarek na tistem delu področja, ki se nanaša na kakovost in čiščenje obstoječih podatkov. Predstavljene so vrste napak, različna ogrodja čiščenja podatkov ter prikaz postopka z združenimi poudarki različnih ogrodij. Narejen je tudi pregled obstoječih programskih rešitev s tega področja. Omenjeno je predstavljeno v prvem, teoretičnem delu naloge. Drugi del predstavlja praktični del, kjer je podan predlog za izboljšanje stanja v organizacijah s pomočjo izdelane programske rešitve – prototipa za realizacijo tistega dela upravljanja s kakovostjo podatkov, ki se nanaša na vzdrževanje pravilnosti podatkov s pomočjo zaznavanja napak v podatkih in možnost njihove odprave. Podan je tudi predlog uporabe rešitve v konkretni organizaciji s predlogom umestitve v obstoječi informacijski sistem z upoštevanjem vodil in principov, ki jih predlaga literatura. V zaključnem delu naloge so podani ključni pristopi, ki bi v organizacijah pripomogli k izboljšanju stanja na tem področju.

• Language: Slovenian
• Keywords: kakovost podatkov, celovitost podatkov, upravljanje kakovosti podatkov, upravljanje podatkov, čiščenje podatkov, informacijska varnost
• Work type: Master's thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2016
• PID: 20.500.12556/RUL-83713
• Publication date in RUL: 23.06.2016

Secondary language

• Language: English
• Title: Data quality management and data cleaning

Abstract

Today´s enterprises are often challenged by managing a large amount of data used in their business operation. Assurance and maintenance of adequate data quality level are important aspects of data quality management due to many reasons. On the one hand, the adequate data quality level represents a competitive advantage, and on the other hand, low data quality level leads to many unpleasant consequences. In the past, frameworks, methodologies, and tools to help ensuring adequate level of data quality were formed. Besides, the question of data quality is discussed in legislation and various standards. Despite that fact, some researches show poor state of data quality in enterprises. A purpose of the thesis is to research and present the area of data quality, and to show subsequent issues of low data quality. The thesis presents consequences as well as reasons of low data quality. It also shows reasons of data quality importance. In addition, it presents standards, legislation, and best practices that deal with the field of data quality. Data quality issues also arise in the field of the Internet of Things, which is an object of many researches lately, therefore, the thesis also presents main issues from that point of view. The main emphasis of the thesis is on the part of the field dealing with data quality and data cleaning. The thesis presents error types, various data cleaning frameworks, and combines their main activities in a consolidated view. Furthermore, the thesis presents an overview of the existing software solutions available on the market to support data cleaning tasks. The aforementioned is introduced in the theoretical part of the thesis. The second part of the thesis represents a practical part, where a proposal for data quality improvement is given using a prototype of a software solution to address a specific part of data quality management, which deals with data accuracy maintenance by sensing errors in data, and the possibility of error elimination (data cleaning). In addition, the thesis proposes installation of the solution in a concrete organisation´s information system by considering principles and rules the literature suggests. In the conclusion, there are essential approaches given to aid the improvement of data quality field in enterprises.

• Keywords: data quality, data integrity, data quality management, DQM, data management, data cleaning, information security