Tag Archives: Flask

Topic 24 – Introduction to Nature Language Processing

Why do I need to learn about nature language processing?

Natural language processing (NLP) has become more and more interesting. Speech recognition, speech synthesis, autonomous driving and chat bots are examples of breakthrough achievements in the field.

Nowadays a key skill of software developer is the ability to use nature language processing algorithms and tools to solve real-world problems related to text, audio, natural language sentences and speech.

What can I do after finishing learning about nature language processing?

You will be to create software that could recognize speech, translate text to speech, translate a sentence from English to French, answer a customer’s question.

That sounds fun! What should I do now?

Please read
– this Daniel Jurafsky and James H. Martin (2014). Speech and Language Processing. Pearson book, and
– this Christopher D. Manning and Hinrich Schiitze (1999). Foundations of Statistical Natural Language Processing. MIT Press book first.

After that please audit these Natural Language Processing Specialization courses and this Stanford CS224N – NLP with Deep Learning, Winter 2023 course (Lecture Notes).

Terminology Review:

  • Natural Language Processing.
  • Text Classification (e.g. Spam Detection).
  • Named Entity Recognition.
  • Chatbots.
  • Speech Processing.
  • Speech Recognition.
  • Speech Synthesis.
  • Machine Translation.
  • Corpus: A body of texts.
  • Token: a word or a number or a punctuation mark.
  • Collocation: compounds (e.g. disk drive), phrasal verbs (e.g. make up), and other stock phrases (e.g. bacon and eggs).
  • Unigram: word.
  • Bigrams: pairs of words that occur commonly.
  • Trigrams: 3 words that occur commonly.
  • N-grams: n words that occur commonly.
  • Hypothesis Testing.
  • t-Test.
  • Likelihood Ratios.
  • Language Model: statistical model of word sequences.
  • Naive Bayes.
  • Hidden Markov Models.
  • Bag-of-Words Model.
  • Term Frequency–Inverse Document Frequency (TF–IDF).
  • Bag-of-n-Grams.
  • One-Hot Representation: You have a vocabulary of n words and you represent each word using a vector that is n bits long, in which all bits are zero except for one bit that is set to 1.
  • Word Embedding (Featurized Representation) is the transformation from words to dense vector.
  • Euclidean Distance, Dot Product Similarity, Cosine Similarity.
  • Embedding Matrix.
  • Neural Language Model.
  • Word2Vec: Skip-Gram Model, Bag-of-Words Model.
  • Negative Sampling.
  • GloVe, Global Vectors.
  • Recurrent Neural Networks.
  • Backpropagation Through Time.
  • Recurrent Neural Net Language Model (RNNLM).
  • Gated Recurrent Unit (GRU).
  • Long Short Term Memory (LSTM).
  • Bidirectional RNN.
  • Deep RNNs.
  • Sequence to Sequence Model.
  • Teacher Forcing.
  • Image Captioning.
  • Greedy Search.
  • Beam Search, Length Normalization.
  • BLEU (BiLingual Evaluation Understudy) Score.
  • ROUGE (Recall-Oriented Understudy for Gisting Evaluation) Score.
  • F1 Score.
  • Minimum Bayes-Risk.
  • Attention Mechanism.
  • Self-Attention (Scaled and Dot-Product Attention): Queries, Keys and Values.
  • Positional Encoding.
  • Masked Self-Attention.
  • Multi-Head Attention.
  • Residual Dropout.
  • Label Smoothing.
  • Transformer Encoder.
  • Transformer Decoder.
  • Transformer Encoder-Decoder.
  • Cross-Attention.
  • Byte Pair Encoding.
  • BERT (Bidirectional Encoder Representations from Transformers).

After finishing learning about natural language processing please click Topic 25 – Introduction to Distributed Systems to continue.



Topic 8 – Introduction to Web Application Development

Why do I need to learn about web application development?

Desktop applications are very powerful and convenient but their development, deployment and maintenance are daunting.

The reason is that the platform dependency makes it very expensive to create a desktop application working on different versions of different operating systems, such as Windows, Linux and Mac OSX.

Deployment and updates of desktop application typically require high privileges access to a computer machine, causing a problem for companies requiring high security.

Fortunately, you can overcome these limitations by creating a web application running on a browser. To create a web application you need to learn about web application development.

What can I do after finishing learning web application development?

You will be able to create web applications like The BBC News, The WordPress Blog or The White House Website.

This is exactly what I want to learn! What should I do now?

Web application development requires a lot of reading. You have to master networking concepts, HTML, CSS, JavaScript, a programming language and a database management system for web. Please read
– this Semmy Purewal (2014). Learning Web App Development. O’Reilly Media book, and
– this Jon Duckett (2011). HTML & CSS – Design and Build Websites. John Wiley & Sons book first to get familiar with web application development.

After that please read
– this Stephen Greig (2013). CSS3 – Pushing the Limits. John Wiley & Sons book and
– this Anne Boehm and Zak Ruvalcaba (2018). Murach’s HTML5 and CSS3. Mike Murach and Associates book to learn in-depth about HTML and CSS.

JavaScript is the language for web development because it is implemented in most of the web browsers.
Please read
– this David Flanagan (2020). JavaScript: The Definitive Guide. O’Reilly Media book, and
– this Kyle Simpson (2015). You Don’t Know JS. O’Reilly Media book, and
– this Jon Duckett (2014). JavaScript and JQuery. Interactive Front-End Web Development. John Wiley & Sons book to master it.
Strong knowledge of JavaScript will ease your web development learning very much.

Single page application is the default front-end choice for most of new web development projects.
Please read
– this Kirupa Chinnathambi (2018). Learning React. Addison-Wesley Professional book, and
– this Alex Banks and Eve Porcello (2020). Learning React Modern Patterns for Developing React Apps. O’Reilly book to learn how to create a single page application (SPA).

After that you will have 4 main options. You can choose one of them. We STRONGLY recommend that you choose only ONE option.

You should NOT learn all of them at the beginning. You could save your time by digging into only one option. After mastering the selected technology, you will realize that all of them are very similar in the sense of use.

One note is that although their concepts are similar to one another but it still takes us much time to learn how to apply an approach to real world web application.

When developing a real world web application, you often use only one or two of these 4 approaches. If you cannot make your own selection then we recommend you
– a combination of the first and second option, or
– a combination of the first and third option, or
– a combination of the first and fourth option.

The first option is PHP world.  Please read
– this Luke Welling and Laura Thomson (2016). PHP and MySQL Web Development. Addison-Wesley Professional book or
– this Robin Nixon (2021). Learning PHP, MySQL & JavaScript. O’Reilly book.
After that depending on your projects you may need to read the books below
Brad Williams et al. (2015). Professional WordPress: Design and Development. Wrox.

The second option is ASP.NET.
Please read
– this Adam Freeman (2020). Pro ASP.NET Core 3: Develop Cloud-Ready Web Applications Using MVC, Blazor, and Razor Pages. Apress book and
– this Andrew Lock (2021). ASP.NET Core in Action. Manning book.

A complementary part for this option is ASP.NET Web Forms that is a technology that you need to master if you are maintaining a legacy project.
Please read this Imar Spaanjaars (2014). Beginning ASP.NET 4.5.1 in C# and VB. Wrox book to learn about ASP.NET Web Forms.

The third option is Java world.
If you are not familiar with Java language then please read
– this Cay S. Horstmann (2019). Core Java. Volume I – Fundamentals. Pearson book, and
– this Cay S. Horstmann (2019). Core Java. Volume II – Advanced Features. Pearson book first.

Then please read
– this Tim Downey (2021). Guide to Web Development with Java – Understanding Website Creation. Springer book or
– this Nicholas S. Williams (2014). Professional Java for Web Applications. John Wiley & Sons book.

After that please read this Mark Heckler (2021). Spring Boot – Up and Running – Building Cloud Native Java and Kotlin Applications. O’Reilly Media book.

The 4th option is Node.js world.
Please read
– this Jonathan Wexler (2019). Get Programming with Node.js. Manning Publications book, and
– this Bruno Joseph D’mello et al. (2017). Web Development with MongoDB and Node. Packt Publishing book.

There are also several other options that you may consider if you are required to learn them for a maintenance project.
These options include
Ruby on Rails, please read this Michael Hartl (2020). The Ruby on Rails Tutorial. Addison-Wesley Professional book,
Flask, please read this Miguel Grinberg (2018). Flask Web Development: Developing Web Applications with Python. O’Reilly Media book, and

If you need to convert a web application from one platform to another or create a web application framework then please read
– this Leon Shklar and Richard Rosen (2009). Web Application Architecture. John Wiley & Sons book, and
– this Leonard Richardson and Mike Amundsen (2013). RESTful Web APIs. O’Reilly Media book.

Terminology Review:

  • Networking
  • HTTP
  • HTML
  • Cascading Style Sheets (CSS)
  • CGI
  • Web Applications
  • Multiple Page Application
  • Single-Page Application (SPA)
  • RESTful APIs
  • gRPC APIs
  • GraphQL APIs
  • XML
  • Web Application Frameworks
  • Portal Frameworks
  • Content Management System (CMS)
  • Payment Gateways

After finishing learning about web application development please click Topic 9 – Introduction to Mobile Application Development to continue.