All posts by admin

Topic 26 – Introduction to Cloud Computing

Why do I need to learn about cloud computing?

Because you will develop software systems that usually leverage cloud services for quick deployment or scalable computation or storage.

What can I do after finishing learning cloud computing?

You will be able to

  • deploy software systems to public clouds,
  • build your private cloud,
  • develop software using cloud plaftforms,
  • develop software using cloud services,
  • leverage cloud services for training and deploying machine learning models,
  • leverage cloud services for big data analytics and reporting.

What should I do now?

Please read
– this Dan C. Marinescu (2022). Cloud Computing – Theory and Practice. Morgan Kaufmann book, and
– this Andreas Wittig and Michael Wittig (2022). Amazon Web Services in Action. Manning Publications book, and
– this Nick Marshall et al. (2019). Mastering VMware vSphere 6.7. Sybex book, and
– this Rakesh Gupta (2020). Salesforce Platform App Builder Certification – A Practical Study Guide. Apress book, and
– this Philip Weinmeister (2019). Practical Salesforce Development Without Code – Building Declarative Solutions on the Salesforce Platform. Apress book, and
– this Tomasz Wiktorski (2019). Data-Intensive Systems – Principles and Fundamentals using Hadoop and Spark. Springer book.

Terminology Review:

  • Software as a Service
  • Multitenancy
  • Infrastructure as a Service
  • Virtual Machines
  • Software-Defined Networking
  • Infrastructure as Code (IaC)
  • Platform as a Service
  • Containers as a Service
  • Function as a Service (Serverless Computing)
  • File Storage
  • Block Storage
  • Object Storage
  • Direct-Attached Storage (DAS)
  • Network-Attached Storage (NAS)
  • Storage Area Network (SAN)
  • GFS
  • Bigtable
  • MapReduce

After finishing learning about cloud computing please click Topic 27 – Introduction to Block Chain to continue.

 

Topic 25 – Introduction to Distributed Systems

Why do I need to learn about distributed systems?

Distributed systems provides foundation for understanding theories and techniques behind cloud computing and block chain technology.

Architectures, protocols and algorithms introduced in distributed systems are necessary for creating complicated software too.

What can I do after finishing learning distributed systems?

You will be able to design software that can

  • tolerate faults,
  • shard data,
  • handle massive number of requests, and
  • perform expensive computation.

You will be prepared to learn about cloud computing and block chain technology.

What should I do now?

Please watch this Distributed Systems, UC Santa Cruz Baskin School of Engineering, 2021 course to get familar with core concepts and protocols.

After that please watch this MIT 6.824, Distributed Systems, Spring 2020 course to learn how to design a large-scale distributed system.

At the same time you can read
– this Maarten van Steen and Andrew S. Tanenbaum (2023). Distributed Systems. Maarten van Steen book, and
– this Martin Kleppmann (2017). Designing Data-Intensive Applications – The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. O’Reilly Media book to solidify your knowledge.

Terminology Review:

  • Fault Tolerance
  • Consistency
  • System Models
  • Failure Detectors
  • Communication
  • Ordering
  • State Machine Replication
  • Primary-Backup Replication
  • Bully Algorithm
  • Ring Election
  • Multi-Leader Replication
  • Leaderless Replication
  • Cristian’s Algorithm
  • Berkeley Algorithm
  • Lamport Clocks
  • Vector Clocks
  • Version Vectors
  • Chain Replication
  • Consensus Algorithms
  • FLP
  • Raft
  • Paxos
  • Viewstamped Replication
  • Zab
  • Consistent Hashing
  • Distributed Transactions
  • ACID
  • Two-Phase Commit
  • Three-Phase Commit
  • Serializability
  • Two-Phase Locking
  • Distributed Locks
  • CAP
  • Consistency Models
  • Linearizability
  • Distributed Architectures
  • Distributed Programming
  • Hadoop
  • Spark
  • Tensorflow
  • PyTorch
  • Kubernetes
  • Bitcoin
  • Smart Contracts

After finishing learning about computer networks please click Topic 26 – Introduction to Cloud Computing to continue.

 

When to Use Cloud Resources

Motivation:

  • You are using on-premises virtual servers and storage for your services.
  • You wonder whether you should move your services to public cloud, such as AWS or Azure or Google Cloud, in order to reduce hardware and maintenance cost.

Suggestions:

  1. It is common that cloud resources for system running 24/7 are much more expensive than private resources.

  2. Cloud resources are suitable for 
    • a very big system that needs to be run only once within limited time, or
    • startup with many consolidated services on single small virtual server, or
    • testing purpose, or
    • a very big company with big budget who want reliable and elastic resources.

       

      Topic 20 – Discrete Mathematics

      Why do I need to learn about discrete mathematics?

      Discrete mathematics is a fundamental tool for understanding many theories and techniques behind artificial intelligence, machine learning, deep learning, data mining, security, digital imagine processing and natural language processing.

      The problem-solving techniques and computation thinking introduced in discrete mathematics are necessary for creating complicated software too.

      What can I do after finishing learning discrete mathematics?

      You will be prepared to learn modern theories and techniques to create modern security, machine learning, data mining, image processing or natural language processing software.

      What should I do now?

      Please read
      – this Kenneth H. Rosen (2012). Discrete Mathematics and Its Applications. McGraw-Hill book and
      – this Alfred V. Aho and Jeffrey D. Ullman (1994). Foundations of Computer Science  book (free online version).

      Alternatively, please watch this MIT 6.042J – Mathematics for Computer Science, Fall 2010 course (Textbook).

      Terminology Review:

      • Statement: An assertion that is either true or false.
      • Mathematical Statements.
      • Mathematical Proof: A convincing argument about the accuracy of a statement.
      • If p, then q. p is hypothesis. q is conclusion.
      • Proposition: A true statement.
      • Theorem: An important proposition.
      • Lemmas: Supporting propositions.
      • Logic: A language for reasoning that contains a collection of rules that we use when doing logical reasoning.
      • Propositional Logic: A logic about truth and falsity of statements.
      • Logic Connectives: Not (Negation), And (Conjunction), Or (Disjunction), If then (Implication), If and only if (Equivalence).
      • Truth Table.
      • Contrapositive of Proposition: The contrapositive of p q is the proposition ¬q ¬p.
      • Modus Ponens: If both P  Q and P hold, then Q can be concluded.
      • Predicate: A property of some objects or a relationship among objects represented by the variables.
      • Quantifier: Tells how many objects have a certain property.
      • Mathematical Induction: Base Case, Inductive Case.
      • Recursion: A Base, An Recursive Step.
      • Sum Example: Annuity.
      • Set.
      • Subset.
      • Set Operations: A ∪ B, A ∩ B, A ⊂ U: A’ = {x : x ∈ U and x ∉ A}, A \ B = A ∩ B’ = {x : x ∈ A and x ∉ B}.
      • Cartesian Product: A × B = {(a; b) : a ∈ A and b ∈ B};
      • A binary relation (or just relation) from X to Y is a subset R ⊆ X × Y. To describe the relation R, we  may list the collection of all ordered pairs (x, y) such that x is related to y by R.
      • A mapping or function f ⊂ A × B from a set A to a set B to be the special type of relation in which for each element a ∈ A there is a unique element b ∈ B such that (a, b) ∈ f.
      • Equivalence Relation.
      • Equivalence Class.
      • Partition.
      • A state machine is a binary relation on a set, the elements of the set are called states, the relation is called the transition relation, and an arrow in the graph of the transition relation is called a transition.
      • Greatest Common Divisor.
      • Division Algorithm.
      • Prime Numbers.
      • The Fundamental Theorem of Arithmetic: Let n be an integer such that n > 1. Then n can be factored as a product of prime numbers. n = p₁p₂ ∙ ∙ ∙ pₖ
      • Congruence: a is congruent to b modulo n if n | (a – b), written a ≡ b (mod n).
      • Fermat’s Little Theorem.
      • Stirling’s Approximation.
      • Probability.
      • Example: The Monty Hall Problem.
      • The Four Step Method: (1) Find the Sample Space (Set of possible outcomes), (2) Define Events of Interest (Subset of the sample space),  (3) Determine Outcome Probabilities, (4) Compute Event Probabilities.
      • A tree diagram is a graphical tool that can help us work through the four step approach when the number of outcomes is not too large or the problem is nicely structured.
      • Example: The Strange Dice.
      • Conditional Probability: P(A|B) = P (A ∩ B) / P(B).
      • A conditional probability P(B|A) is called a posteriori if event B precedes event A in time.
      • Example: Medical Testing.
      • Independence: P(B|A) = P(B)  or P(A∩B) = P(A) · P(B).
      • Mutual Independence: The probability of each event is the same no matter which of the other events has occurred.
      • Pairwise Independence: Any two events are independent.
      • Example: The Birthday Problem.
      • The birthday paradox refers to the counterintuitive fact that only 23 people are needed for that probability to exceed 50%, for 70 people: P = 99.9%.
      • Bernoulli Random Variable (Indicator Random Variable): f: Ω {1, 0}.
      • Binomial Random Variable: A number of successes in an experiment consisting of n trails. P (X = x) = [(n!)/((x!) · (n-x)!))]pˣ(1 − p)ⁿ − ˣ
      • Expectation (Average, Mean). E = Sum(R(w) · P(w)) = Sum(x · P(X = x)).
      • Median P(R < x) ≤ 1/2 and P(R>x) < 1/2.
      • Example: Splitting the Pot.
      • Mean Time to Failure: If a system independently fails at each time step with probability p, then the expected number of steps up to the first failure is 1/p.
      • Linearity of Expectation.
      • Example: The Hat Check Problem.
      • Example: Benchmark: E(Z/R) = 1.2 does NOT mean that E(Z) = 1.2E(R).
      • Variance: var(X) = E[(X−E[X])²].
      • Kurtosis: E[(X−E[X])⁴].
      • Markov’s Theorem: P(R ≥ x) ≤ E(R)/x (R > 0, x > 0).
      • Chebyshev’s Theorem: P(|R – E(R)| ≥ x) ≤ var(R)/x². Boundary of the probability of deviation from the mean.
      • The Chernoff Bound: P(T ≥ c·E(T)) ≤ e−ᶻ·ᴱ⁽ᵀ⁾, where z = c·lnc − c + 1, T = Sum(Tᵢ),  0 ≤ Tᵢ ≤ 1.

      After finishing learning about discrete mathematics please click Topic 21 – Introduction to Computation and Programming using Python to continue.

       

      Topic 2 – Introduction to Computer Networks

      Why do I need to learn about computer networks?

      Because you will develop software system that usually connects with other software systems via various networks.

      What can I do after finishing learning computer networks?

      You will be able to set up various software systems such as Domain Name System, Active Directory System, Electronic Mail, File Transfer Protocol System, Remote Desktop Services, File Services, HTTP Services.

      You will be prepared to learn about network programming, game development, web application development, and distributed systems and blockchain.

      What should I do now?

      Please audit this The Bits and Bytes of Computer Networking course and complete all the quizzes.

      Alternatively, you can read
      – this Andrew S. Tanenbaum and David J. Wetherall (2021). Computer Networks. Pearson Education book, and
      – this James F. Kurose and Keith W. Ross (2021). Computer Networking: A Top-Down Approach. Pearson book.

      After that please read
      – this Brian Svidergol et al. (2018). Mastering Windows Server 2016. Wiley book, and
      – this Larry L. Peterson and Bruce S. Davie (2021). Computer Networks: A Systems Approach. Morgan Kaufmann book.

      Terminology Review:

      • Computer Networking.
      • Computer Networks, Peer-to-Peer Systems, Local Area Networks, Wide Area Networks, Virtual Private Networks, ISP Networks, The Internet.
      • Network Software, Distributed Systems, World Wide Web, Network Protocols.
      • The OSI Reference Model: The Physical Layer, The Data Link Layer, The Network Layer, The Transport Layer, The Session Layer, The Presentation Layer, The Application Layer.
      • The TCP/IP Reference Model: The Link Layer, The Internet Layer, The Transport Layer, The Application Layer.
      • The TCP/IP 5-Layer Model: The Physical Layer, The Data Link Layer, The Network Layer, The Transport Layer, The Application Layer.
      • Network Interface Cards, RJ45 Ports and Plugs, Cables, Hubs, Switches, Routers, Servers, Clients, Nodes.
      • Bit, Octet (Byte), Modulation, Line Coding, Twisted Pair Cables, Simplex Communication, Duplex Communication, Full-Duplex, Half-Duplex.
      • Collision Domain, Ethernet, Carrier-Sense Multiple Access with Collision Detection (CSMA/CD), MAC Address.
      • Unicast, Broadcast, Multicast.
      • Data Packet, Ethernet Frame, Virtual LAN (VLAN), VLAN Header.
      • First-in-First-Out (FIFO).
      • IPv4 Addresses, IIPv4 Datagrams, IPv4 Address Classes, Address Resolution Protocol (ARP), Subnet Masks, CIDR (Classless Inter-Domain Routing).
      • Routing Tables, Autonomous System, Interior Gateway Protocols,  Exterior Gateway Protocols, Distance Vector Routing Protocols, Link State Routing Protocols, Core Internet Routers, Border Gateway Protocol (BGP), Non-Routable Address Space.
      • Multiplexing, Demultiplexing, Ports.
      • TCP Segment, TCP Control Flags, Three-way Handshake, Four-way Handshake, Transmission Control Protocol (TCP), TCP Socket, TCP Socket States.
      • Connection-Oriented Protocols, Connectionless Protocols.
      • User Datagram Protocol (UDP).
      • Firewall.
      • Network Address Translation.
      • Frames, Packets, Messages.
      • Network Socket.
      • Transport Service Primitives: LISTEN, CONNECT, SEND, RECEIVE, DISCONNECT.
      • Domain Name System (DNS).
      • Electronic Mail, SMTP Protocol.
      • File Transfer Protocol System.
      • Remote Desktop Services.
      • File Services.
      • HTTP Services.
      • Time Services.
      • Short Message Service (SMS).
      • Public Switched Telephone Network (PSTN), Plain Old Telephone Service (POTS), Modems, Dial-up (Phone Lines), Usenet.
      • Broadband, T-Carrier Technologies, Digital Subscriber Line (DSL, Phone Lines), Asymmetric Digital Subscriber Line (ADSL), Symmetric Digital Subscriber Line (SDSL), High Bit-Rate Digital Subscriber Line (HDAL), Digital Subscriber Line Access Multiplexers (DSLAM).
      • Cable Broadband (Television Lines), Cable Modems, Cable Modem Termination System (CMTS).
      • Fiber to the X (FTTX), Fiber to the Neighborhood (FTTN), Fiber to the Building (FTTB), Fiber to the Home (FTTH), Fiber to the Premises (FTTP), Optical Network Terminator.
      • Point to Point Protocol (PPP), Network Control Protocol (NCP), Link Control Protocol (LCP), Point to Point Protocol over Ethernet (PPPoE).

      After finishing learning about computer networks please click Topic 3 – Introduction to Programming to continue.

       

      How to Reduce Salesforce Licenses Cost

      Motivation:

      • You are using Salesforce for your daily work. It works very well but the cost is too expensive.
      • You want to find  a way to reduce Salesforce licenses cost.

      Suggestions:

      1. Salesforce cost depends on a selected Edition and number of user licenses.

      2. An edition is selected based on features that an ORG needs.

      For example, an ORG that needs the API and Flows feature for custom development has to choose Professional or Enterprise edition.

      The number Flows are very limited for Professional edition (5 flows).

      However the max number of flows of the Enterprise edition is 2000 that may be too many for a small business too.

      This may be Salesforce intention that they want customers to use Enterprise edition.

      3. A license price depends on the selected edition. For example, an ORG needs to  pay $150 /user/month for the Enterprise edition.

      4. A User License may be Salesforce License or Salesforce Platform License. A Profile defines permissions. A Profile with login is attached to one User License (except the case that the Profile is used for Community portal users).

      5. Assume that an ORG may use 13 User Licenses: 5 System Administrators (5 Salesforce Licenses) and 8 Platform Users (8 Salesforce Platform Licenses).

      In order to reduce the cost the ORG may consider some strategies below.

      • The ORG may use shared account for all the employees who have the same responsibilities (tasks) if possible.
      • The ORG may retain only 2 Salesforce licenses: One is for API access and development, and the other is for administrative tasks.
      • The ORG may change appropriate licenses from Salesforce license to Platform license if they do not use the built-in Leads, Opportunities, Forecasts, Cases, and Solutions.
      • The ORG should disable all unused accounts.
      • The ORG may develop Community portal for some user types and move all these users from Platform license to Community license.
      • If you are using Salesforce development platform then try moving your custom development features to another cheaper development platform, such as Zoho Creator.
      • If you are using Salesforce CRM as is then try moving your business workflows and data to another cheaper provider, such as EspoCRM.

       

        Topic 24 – Introduction to Nature Language Processing

        Why do I need to learn about nature language processing?

        Natural language processing (NLP) has become more and more interesting. Speech recognition, speech synthesis, autonomous driving and chat bots are examples of breakthrough achievements in the field.

        Nowadays a key skill of software developer is the ability to use nature language processing algorithms and tools to solve real-world problems related to text, audio, natural language sentences and speech.

        What can I do after finishing learning about nature language processing?

        You will be to create software that could recognize speech, translate text to speech, translate a sentence from English to French, answer a customer’s question.

        That sounds fun! What should I do now?

        Please read
        – this Daniel Jurafsky and James H. Martin (2014). Speech and Language Processing. Pearson book, and
        – this Christopher D. Manning and Hinrich Schiitze (1999). Foundations of Statistical Natural Language Processing. MIT Press book first.

        After that please audit these Natural Language Processing Specialization courses and this Stanford CS224N – NLP with Deep Learning, Winter 2023 course (Lecture Notes).

        Terminology Review:

        • Natural Language Processing.
        • Text Classification (e.g. Spam Detection).
        • Named Entity Recognition.
        • Chatbots.
        • Speech Processing.
        • Speech Recognition.
        • Speech Synthesis.
        • Machine Translation.
        • Corpus: A body of texts.
        • Token: a word or a number or a punctuation mark.
        • Collocation: compounds (e.g. disk drive), phrasal verbs (e.g. make up), and other stock phrases (e.g. bacon and eggs).
        • Unigram: word.
        • Bigrams: pairs of words that occur commonly.
        • Trigrams: 3 words that occur commonly.
        • N-grams: n words that occur commonly.
        • Hypothesis Testing.
        • t-Test.
        • Likelihood Ratios.
        • Language Model: statistical model of word sequences.
        • Naive Bayes.
        • Hidden Markov Models.
        • Bag-of-Words Model.
        • Term Frequency–Inverse Document Frequency (TF–IDF).
        • Bag-of-n-Grams.
        • One-Hot Representation: You have a vocabulary of n words and you represent each word using a vector that is n bits long, in which all bits are zero except for one bit that is set to 1.
        • Word Embedding (Featurized Representation) is the transformation from words to dense vector.
        • Euclidean Distance, Dot Product Similarity, Cosine Similarity.
        • Embedding Matrix.
        • Neural Language Model.
        • Word2Vec: Skip-Gram Model, Bag-of-Words Model.
        • Negative Sampling.
        • GloVe, Global Vectors.
        • Recurrent Neural Networks.
        • Backpropagation Through Time.
        • Recurrent Neural Net Language Model (RNNLM).
        • Gated Recurrent Unit (GRU).
        • Long Short Term Memory (LSTM).
        • Bidirectional RNN.
        • Deep RNNs.
        • Sequence to Sequence Model.
        • Teacher Forcing.
        • Image Captioning.
        • Greedy Search.
        • Beam Search, Length Normalization.
        • BLEU (BiLingual Evaluation Understudy) Score.
        • ROUGE (Recall-Oriented Understudy for Gisting Evaluation) Score.
        • F1 Score.
        • Minimum Bayes-Risk.
        • Attention Mechanism.
        • Self-Attention (Scaled and Dot-Product Attention): Queries, Keys and Values.
        • Positional Encoding.
        • Masked Self-Attention.
        • Multi-Head Attention.
        • Residual Dropout.
        • Label Smoothing.
        • Transformer Encoder.
        • Transformer Decoder.
        • Transformer Encoder-Decoder.
        • Cross-Attention.
        • Byte Pair Encoding.
        • BERT (Bidirectional Encoder Representations from Transformers).

        After finishing learning about natural language processing please click Topic 25 – Introduction to Distributed Systems to continue.

         

         

        Topic 23 – Introduction to Computer Vision

        Why do I need to learn about computer vision?

        Computer vision has become more and more interesting. Image recognition, autonomous driving, and disease detection are examples of breakthrough achievements in the field.

        Nowadays a key skill that is often required from a software developer is the ability to use computer vision algorithms and tools to solve real-world problems related to images and videos.

        What can I do after finishing learning about applied computer vision?

        You will be to create software that could recognize recognize a face or transform a picture of young person to old person.

        That sounds fun! What should I do now?

        Please read
        – this Rafael C. Gonzalez and Richard E. Woods (2018). Digital Image Processing. 3rd Edition. Pearson book, and
        – this Richard Szeliski (2022). Computer Vision: Algorithms and Applications. Springer book.

        At the same time, please
        – audit these Deep Learning Specialization courses and
        – read this Francois Chollet (2021). Deep Learning with Python. Manning Publications book, and
        – this Michael A. Nielsen (2015). Neural Networks and Deep Learning. Determination Press book.

        After that please read this Ian Goodfellow et al. (2016). Deep Learning. The MIT Press book.

        Terminology Review:

        • Deep Learning.
        • Artificial Neural Networks.
        • Filter: 2-dimensional matrix commonly square in size containing weights shared all over the input space.
        • The Convolution Operation: Element-wise multiply, and add the outputs.
        • Stride: Filter step size.
        • Convolutional Layers.
        • Feature Maps.
        • Pooling.
        • Convolutional Neural Networks (CNNs).
        • Object Detection.
        • Face Recognition.
        • YOLO Algorithm.
        • Latent Variable.
        • Autoencoders.
        • Variational Autoencoders.
        • Generators.
        • Discriminators.
        • Generative Adversarial Networks (GANs).
        • CycleGAN.
        • Neural Style Transfer.

        After finishing learning about computer vision please click Topic 24 – Introduction to Nature Language Processing to continue.