Agile Data Science - Data Enrichment

Data enrichment refers to a range of procedures used to upgrade, refine and improve raw data. It refers to helpful data transformation (raw data to valuable information). The process of data enrichment focusses on making data a valuable data asset for modern business or enterprise.

The most common data enrichment process includes amendment of spelling mistakes or typographical errors in database through utilization of specific decision algorithms. Data enrichment tools add useful information to simple data tables.

Consider the following code for spell correction of words −

import re
from collections import Counter
def words(text): return re.findall(r'\w+', text.lower())
WORDS = Counter(words(open('big.txt').read()))

def P(word, N=sum(WORDS.values())):
   "Probabilities of words"
   return WORDS[word] / N
def correction(word):
   "Spelling correction of word"
   return max(candidates(word), key=P)
def candidates(word):
   "Generate possible spelling corrections for word."
   return (known([word]) or known(edits1(word)) or known(edits2(word)) or [word])
def known(words):
   "The subset of `words` that appear in the dictionary of WORDS."
   return set(w for w in words if w in WORDS)
def edits1(word):
   "All edits that are one edit away from `word`."
   letters = 'abcdefghijklmnopqrstuvwxyz'
   splits = [(word[:i], word[i:]) for i in range(len(word) + 1)]
   deletes = [L + R[1:] for L, R in splits if R]
   transposes = [L + R[1] + R[0] + R[2:] for L, R in splits if len(R)>1]
   replaces = [L + c + R[1:] for L, R in splits if R for c in letters]
   inserts = [L + c + R for L, R in splits for c in letters]
   return set(deletes + transposes + replaces + inserts)
def edits2(word):
   "All edits that are two edits away from `word`."
   return (e2 for e1 in edits1(word) for e2 in edits1(e1))

In this program, we will match with “big.txt” which incorporates corrected words. Words coordinate with words included in text file and print the suitable results accordingly.


The above code will generate the following output −


Input your Topic Name and press Enter.