Tag Archives: Python

Topic 24 – Introduction to Nature Language Processing

Why do I need to learn about nature language processing?

Natural language processing (NLP) has become more and more interesting. Speech recognition, speech synthesis, autonomous driving and chat bots are examples of breakthrough achievements in the field.

Nowadays a key skill of software developer is the ability to use nature language processing algorithms and tools to solve real-world problems related to text, audio, natural language sentences and speech.

What can I do after finishing learning about nature language processing?

You will be to create software that could recognize speech, translate text to speech, translate a sentence from English to French, answer a customer’s question.

That sounds fun! What should I do now?

Please read
– this Daniel Jurafsky and James H. Martin (2014). Speech and Language Processing. Pearson book, and
– this Christopher D. Manning and Hinrich Schiitze (1999). Foundations of Statistical Natural Language Processing. MIT Press book first.

After that please audit these Natural Language Processing Specialization courses and this Stanford CS224N – NLP with Deep Learning, Winter 2023 course (Lecture Notes).

Terminology Review:

  • Natural Language Processing.
  • Text Classification (e.g. Spam Detection).
  • Named Entity Recognition.
  • Chatbots.
  • Speech Processing.
  • Speech Recognition.
  • Speech Synthesis.
  • Machine Translation.
  • Corpus: A body of texts.
  • Token: a word or a number or a punctuation mark.
  • Collocation: compounds (e.g. disk drive), phrasal verbs (e.g. make up), and other stock phrases (e.g. bacon and eggs).
  • Unigram: word.
  • Bigrams: pairs of words that occur commonly.
  • Trigrams: 3 words that occur commonly.
  • N-grams: n words that occur commonly.
  • Hypothesis Testing.
  • t-Test.
  • Likelihood Ratios.
  • Language Model: statistical model of word sequences.
  • Naive Bayes.
  • Hidden Markov Models.
  • Bag-of-Words Model.
  • Term Frequency–Inverse Document Frequency (TF–IDF).
  • Bag-of-n-Grams.
  • One-Hot Representation: You have a vocabulary of n words and you represent each word using a vector that is n bits long, in which all bits are zero except for one bit that is set to 1.
  • Word Embedding (Featurized Representation) is the transformation from words to dense vector.
  • Euclidean Distance, Dot Product Similarity, Cosine Similarity.
  • Embedding Matrix.
  • Neural Language Model.
  • Word2Vec: Skip-Gram Model, Bag-of-Words Model.
  • Negative Sampling.
  • GloVe, Global Vectors.
  • Recurrent Neural Networks.
  • Backpropagation Through Time.
  • Recurrent Neural Net Language Model (RNNLM).
  • Gated Recurrent Unit (GRU).
  • Long Short Term Memory (LSTM).
  • Bidirectional RNN.
  • Deep RNNs.
  • Sequence to Sequence Model.
  • Teacher Forcing.
  • Image Captioning.
  • Greedy Search.
  • Beam Search, Length Normalization.
  • BLEU (BiLingual Evaluation Understudy) Score.
  • ROUGE (Recall-Oriented Understudy for Gisting Evaluation) Score.
  • F1 Score.
  • Minimum Bayes-Risk.
  • Attention Mechanism.
  • Self-Attention (Scaled and Dot-Product Attention): Queries, Keys and Values.
  • Positional Encoding.
  • Masked Self-Attention.
  • Multi-Head Attention.
  • Residual Dropout.
  • Label Smoothing.
  • Transformer Encoder.
  • Transformer Decoder.
  • Transformer Encoder-Decoder.
  • Cross-Attention.
  • Byte Pair Encoding.
  • BERT (Bidirectional Encoder Representations from Transformers).

After finishing learning about natural language processing please click Topic 25 – Introduction to Distributed Systems to continue.



How to Completely Uninstall Python on macOS


You have an issue with a Python version (e.g. the 3.7 version) and want to install another version (e.g. the 3.6 version). In order to install a fresh version you need to uninstall the current version first.


Assume that the current version is 3.7. Replace 3.7 with another version that you installed.

Follow these 3 steps below.

1. Remove the third-party Python 3.7 framework

sudo rm -rf /Library/Frameworks/Python.framework/Versions/3.7

2. Remove the Python 3.7 applications directory

sudo rm -rf "/Applications/Python 3.7"

3. Remove the symbolic links, in /usr/local/bin, that point to this Python version. Ensure that the links exit using below command

ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/3.7'

then run the following commands to remove all the links:

cd /usr/local/bin/
ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/3.7' | awk '{print $9}' | tr -d @ | xargs rm

Installing a new version:

  1. Download a version from https://www.python.org/downloads/mac-osx/, double click the file and follow the instructions.
  2. Verify installation: python3 –version
  3. Install Homebrew from https://brew.sh
  4. Install virtualenv
pip3 install virtualenv
pip3 install virtualenvwrapper

5.  Create and activate a virtual environment



python3 -m virtualenv


source bin/activate

Topic 21 – Introduction to Computation and Programming using Python

Why do I need to learn about computation and programming using Python?

Computational thinking and Python are fundamental tools for understanding many modern theories and techniques such as artificial intelligence, machine learning, deep learning, data mining, security, digital imagine processing and natural language processing.

What can I do after finishing learning about computation and programming using Python ?

You will be prepared to learn modern theories and techniques to create modern security, machine learning, data mining, image processing or natural language processing software.

That sounds useful! What should I do now?

Please read this John V. Guttag (2013). Introduction to Computation and Programming using Python. 2nd Edition. The MIT Press book.

Alternatively, please watch
– this 6.0001 Introduction to Computer Science and Programming in Python. Fall 2016 course (Lecture Notes) and

– this MIT 6.0002 Introduction to Computational Thinking and Data Science, Fall 2016 course (Lecture Notes).

Terminology Review:

  • Big O notation.
  • Monte Carlo Simulation.
  • Random Walk.
  • K-means Clustering.
  • k-Nearest Neighbors Algorithm.

After finishing reading the book please click Topic 22 – Introduction to Machine Learning to continue.


Topic 5 – Object-Oriented Programming

Why do I need to learn about object-oriented programming?

Because most of modern software are written using object-oriented programming languages. Using object-oriented programming as a tool and more importantly as a problem solving approach will save you a lot of time and cost when you create modern software systems.

What can I do after finishing learning object-oriented programming?

You will be able to write a program to allow a user
– to draw a picture, save it, then load it, or
– to play ping-pong game, or
– to store and search for an employee’s CV.

Alright! What should I do now?

Please read this Jim Keogh and Mario Giannini (2004). OOP Demystified. McGraw-Hill book.

Is that all?

Unfortunately, the book is not sufficient for writing real world object-oriented software. Although the book already gives you the fundamental concepts of object-oriented programming the examples are only to illustrate the concepts, not to demonstrate real world situations.

We recommend that you read it first because it introduces basic concepts of object-oriented programming very well. It helps you to distinguish object-oriented concepts from procedural programing concepts very clearly so that you could read other books to dig into real world object-oriented programming in a proper way.

Please read this RB Whitaker (2017). The C# Player’s Guide. Starbound Software book to learn how to apply object-oriented concepts to creating real world software using a specific object-oriented programming language (i.e. C# language).

After that please take a quick look at
– this Joseph Albahari (2023). C# 12 in a Nutshell – The Definitive Reference. O’Reilly Media book or
– this Herbert Schildt (2010). C# 4.0 The Complete Reference. McGraw Hill book so that you can refer to a specific topic that needs more study when developing real world software.

I hear that there are many object-oriented programming languages such as C++, C#, Java, PHP, Objective-C, Swift, Python, Ruby.
How many object-oriented programming languages do I need to know?

You should learn as many as possible. However, in this stage, we recommend that you learn only C++ or C# or Java.

C++ is the most used language for creating software systems that need high performance, including games, operating systems, compilers, database management systems, web browsers, graphics editors, medical and engineering applications.

Java and C# are the most used languages for creating enterprise systems.

If you are required or prefer to learn C++, please read this Bjarne Stroustrup (2013). The C++ Programming Language. Pearson Education book.

If you are required or prefer to learn Java, please read this Herbert Schildt (2019). Java: The Complete Reference. McGraw-Hill Education book.

After that please read
– this Matt Weisfeld (2019). The Object-Oriented Thought Process. Pearson Education book and
– this Bertrand Meyer (1997). Object-Oriented Software Construction. Prentice Hall book to get deeper understanding about object-oriented concepts and learn how to design software using object-oriented approach more efficiently.

Terminology Review:

  • Abstraction.
  • Encapsulation.
  • Class-based Inheritance.
  • Prototype-based Inheritance.
  • Polymorphism.
  • Types.
  • Method Variables (C++ Function Pointers, C# Delegates).
  • Concurrency.
  • Collections.
  • Object-Oriented Analysis.
  • Object-Oriented Design.
  • Object-Oriented Programming.

After finishing learning about object-oriented programming please click Topic 6 – Introduction to Windows Programming to continue.

Topic 3 – Introduction to Programming

Why do I need to know a programming language?

Because you will develop your software using a programming language.
A programming language is a language that both you and a computer will understand.
It’s a tool for you to express what you want a computer to do.
It’s a tool for you to write instructions of your computer programs.
Without knowledge of a programming language, you cannot develop any software.

I hear that there are many programming languages such as C, C++, C#, Objective-C, Java, Swift, Ruby, Python, JavaScript, PHP, F#, Kotlin,  Clojure, Go, or Lisp.
How many programming languages should I know?

As many as possible. However, at this stage, you should learn about structure of a computer program and a procedure language (e.g. C or C++) first.

What can I do after finishing learning programming language?

You will be able to write a program to tell a computer
– to do a calculation, or
– to count for occurrence of a string, or
– to store and search for a string, or
– to sort a list of numbers, or
– to display an image.

What should I do now?

Please read this Stephen Prata (2011). C Primer Plus. Pearson book.

Alternatively, you can read

– this Stephen Prata (2012). C++ Primer Plus. Pearson book (from Chapter 1 to Chapter 9), or

– this John M. Zelle (2017). Python Programming: An Introduction to Computer Science. Franklin, Beedle & Associates book (from Chapter 1 to Chapter 9), or

– this Charles Severance (2013). Python for Informatics: Exploring Information book (from Chapter 1 to Chapter 11), or

– this Chris Pine (2013). Learn to Program: The Facets of Ruby Series. The Pragmatic Bookshelf book (from Chapter 1 to Chapter 11), or

– this Alan A. A. Donovan and Brian W. Kernighan (2015). The Go Programming Language. Addison-Wesley Professional book, and this Nathan Youngman and Roger Peppe (2018). Get Programming with Go. Manning Publications book.

After that if you would like to study system programming in depth then please read

– this Brian W. Kernighan and Dennis M. Ritchie (2016). The C Programming Language. Prentice Hall book, or

– this Michael Kerrisk (2018). The Linux Programming Interface. No Starch Press book.

After that if you would like to get vocabularies and in-depth knowledge about programming, especially the relationship between mathematics and programming, and ideas of functional programming and logic programming then please read

– this Harold Abelson and Gerald Jay Sussman (1996). Structure and Interpretation of Computer Programs. The MIT Press book, or

– these Course Notes.

Terminology Review:

  • Computer Programs.
  • Compilers.
  • Just-In-Time Compilers.
  • Interpreters.
  • Structured Programming.
  • Procedural Programming.
  • Object-Oriented Programming.
  • Functional Programming.
  • Declarative Programming.
  • Logic Programming.
  • Primitive Types: int, float, double, string, date/time, null.
  • Basic Constructs: variables & assignments, pointers, declarations, functions, macro.
  • Control Flow: if/else, while, switch, for, break, continue, recursion, exception, parallelism, signal, jump.
  • User-defined Types: struct, class, type.
  • Data Structures: Arrays, Lists, Linked Lists.
  • Object-Oriented Programming: abstraction, encapsulation, class-based inheritance, prototype-based inheritance, polymorphism.
  • Functional Programming: first-class and higher-order functions, no side effects, recursion.
  • Modularity: files, packages, namespaces, libraries, modules.
  • Concurrency: processes, threads, locks, channels, timers, callbacks, events, promises, event loop.
  • Type system: static type, dynamic type.
  • Error handling: try/catch.
  • Metaprogramming: reflection, template programming, DSL.
    • Clojure.

    After finishing learning about programming languages please click Topic 4 – Introduction to Algorithms and Data Structures to continue.