Category Archives: Software Construction

How to Use MSBuild

1. Fixing “Microsoft.WebApplication.targets was not found” issue.
    • Download vs_BuildTools.exe.
    • Open Command Prompt.
    • Execute the command below.
      vs_BuildTools.exe --add Microsoft.VisualStudio.
      Workload.WebBuildTools --passive
    • Open .csproj or .vbproj file using Notepad.
    • Find Project element and set
      <Project DefaultTargets="Build" ToolsVersion="17.0">
    • Find VisualStudioVersion element and set
      <VisualStudioVersion Condition="'$(VisualStudioVersion)' == ''">17.0</VisualStudioVersion>
    • Find Import Project=”$(MSBuildExtensionsPath32) … Microsoft.WebApplication.targets text and set
      <Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v17.0\WebApplications\Microsoft.WebApplication.targets" Condition="false" />

       

     

    2. Finding build errors.
    • Execute the command below.
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin\MSBuild.exe" /property:Configuration=Release "YourSolution.sln" -restore -p:RestorePackagesConfig=true > Build.log

       

    • Open Build.log file and search for “errors” text.

     

    How to Use nuget

    1. Downloading packages to a build server
    Motivation:

    You have a .NET solution that uses NuGet package manager.

    You have many files in the packages folder and want them to be downloaded to your build server.

    Solution:
    • Download Windows x86 Commandline.
    • Copy nuget.exe to an folder and add the folder path to Windows PATH.
    • Open Command Prompt and execute the commands below.
      cd D:\Build\Source
      nuget restore YourSolutionFileName.sln
      
    2. Downloading (Restoring) packages using MSBuild
    • Execute the commands below.
      "C:\Program Files\Microsoft Visual Studio\2022\Professional\Msbuild\Current\Bin\MSBuild.exe" "../YourSolution.sln" -restore -p:RestorePackagesConfig=true
      
    3. Updating packages for a solution using VS Studio Package Manager Console
    • Open Tools > NuGet Package Manager >  Package Manager Console.
    • Execute the commands below.
      Update-Package
    4. Reinstalling packages for a solution using VS Studio Package Manager Console
    • Open Tools > NuGet Package Manager >  Package Manager Console.
    • Execute the commands below.
      Update-Package –reinstall
      
    5. Configuring package restore when building a project using VS Studio :
    • Open Tools > NuGet Package Manager >  Package Manager Settings.
    6. Reverting PackageReference project to packages.config:
    • Copy and  overwrite project file from backup created by Visual Studio or from source control system.
    • Delete all obj and bin files.
    7. Fixing “Could not load file or assembly…” issue:
    • Example issue:
      Could not load file or assembly 'Microsoft.IdentityModel.Tokens.Saml, Version=8.0.2.0
    • Open Visual Studio > Solution > Project > References > Microsoft.IdentityModel.Tokens.Saml > Properties and identify file or assembly version, e.g. 8.1.2.0.
    • Open Visual Studio > Solution > Project > web.config, search for Microsoft.IdentityModel.Tokens.Saml and modify the version for the file or assembly. Example:
      <bindingRedirect oldVersion="0.0.0.0-8.1.2.0" newVersion="8.1.2.0" />
    • Repeat the process for all the files or assemblies.

     

    How to Automate Code Signing using EV Code Signing Certificate

    Motivation

    Your software installer (e.g. an EXE or MSI file) is warned by Microsoft Defender. You want to remove the warning so that your users can trust your software.

    Solution

    You need to sign your software installer using an EV code signing certificate.

    Terminologies

    SSL certificate is a cryptographic certificate (technically named X.509) issued by a Certificate Authority as a file for encrypting and decrypting data between a client and server.
    SSL certificate is usually valid for one domain in 1 year.

    Code singing certificate is a cryptographic certificate (technically named X.509) issued by a Certificate Authority as a file for placing a digital signature on a file, program, or software.
    Code signing certificate is usually valid for signing file, program or software from 1 year to 3 years.
    Code signing certificate does not relate to domain.

    EV code signing certificate is a code singing certificate (technically named X.509) that is issued on USB or HSM or Cloud Signing Service (not a file) by a Certificate Authority for placing a digital signature on a file, program, or software to bypass MS Defender warning.
    EV code signing certificate is usually valid for signing file, program or software from 1 year to 3 years.
    EV code signing certificate does not relate to domain.
    When signing file, program or software using an EV code signing beside presence of USB or HSM or Cloud Signing Service an additional one-time password (OTP) is usually required for security.

    EV Code Signing Certificate, USB, HSM, and Cloud Code Signing Service

    The core part of a certificate is a secret string called private key.
    All the certificates have this same core part. The difference is just where this string will be stored.

    – This secret string can be stored in a machine/server (standard certificate) and can be exported and imported to other machines/servers, e.g. via PFX files with private keys.

    – This secret string can be stored in a USB (this is a requirement for EV certificate). In this case, it can be imported to the USB but then could NOT be exported from the USB and imported to other locations.

    – This secret string can be stored in a HSM (hardware security module, usually a physical card with cryptoprocessor chip or a certified virtual machine, USB can be considered a HSM too) (this is a requirement for EV certificate). In this case, it can be imported to the HSM but then could NOT be exported from the HSM and imported to other locations.

    – This secret string can be stored in a Cloud Code Signing Service (this is actually a specific certified HSM virtual machine) (this is a requirement for EV certificate). In this case, it can be imported to the Cloud Code Signing Service but then could NOT be exported from the Cloud Code Signing Service and imported to other locations.

    If we have 3 physical servers and we use the USB storage approach for EV Code Signing certificate then we need 3 USB sticks (duplicates) that contain the same certificate.

    If we use HSM or Cloud Code Signing Service then we just need one HSM or one Cloud Code Signing Service account.

    Typical steps to sign code using EV code signing certificate

    – Purchase an EV code signing certificate.

    – Retrieve the USB containing private key and temporary token password.

    – Install software to connect to the USB, e.g. SafeNet Authentication Client, and change the temporary token password.

    – Find and copy the SignTool.exe utility from Visual Studio Tools to a location.

    – Enable Single Logon feature of SafeNet Authentication Client on Client Settings > Advanced so that we can pass the EV certificate password to SignTool.

    – On SafeNet Authentication Client, right click the EV certificate and export the public certificate (.CER file).

    – On SafeNet Authentication Client, click the EV certificate and retrieve the key container name of the EV certificate.

    – Sign the software installer using the command below.

    SignTool sign /f myCert.cer /csp "eToken Base Cryptographic Provider" /k "[{{TokenPasswordHere}}]=KeyContainerNameHere" myFile.msi /tr http://timestamp.digicert.com /td sha256 /fd sha256

    – Verify the signing using the command below.

    SignTool verify /pa myFile.msi

     

     

    How to Automate Cleaning Up a .NET Solution

    Motivation:

    You have a .NET solution with many projects.
    You need to deliver the solution to a client very frequently.
    You need to clean up all intermediate folders and files to reduce the package size and make the result look tidy.
    You want to automate this process to reduce cleaning up time.

    Solution:

    1. In Visual Studio, review Project Build Order… and a last project in the list.
    2. Right click the project and select Properties.
    3. Click on Build Events.
    4. Enter the following commands to the Post-build event command line text box:
      rd /s /q $(ProjectDir)obj
      cd $(TargetDir)
      del *.config
      del *.pdb
      del *.xml

      rd /s /q $(ProjectDir)obj: Remove the obj folder in the project directory.
      cd $(TargetDir): Move to the output directory.
      del *.config: Delete all the config files in the output directory.
      del *.pdb: Delete all the pdb files in the output directory.
      del *.xml: Delete all the xml files in the output directory.

      You can modify these commands or add new commands for your specific purpose.

    5. Save the project.
    6. Repeat the process for some other projects in the solution if necessary. Typically you only need to clean up intermediate folders and files for the some projects in the bottom of the build order list. You may use the Dependencies tab to change the build order for some projects.

    Cleaning a folder manually:

    In order to delete all the obj folders recursively you can

      1. Create DeleteObjFolders.bat file inside the same folder that contains your solution file (i.e. *.sln)
      2. Paste content below to the DeleteObjFolders.bat file:
      @echo off
      @echo Deleting all OBJ folders...
      for /d /r . %%d in (obj) do @if exist "%%d" rd /s/q "%%d"
      @echo OBJ folders successfully deleted :) Press any key to close the window.
      pause > nul

      3. Execute the DeleteObjFolders.bat file.

     

    How to Quickly Capture and Analyze Requirements for Building a Prototype or Proof of Concept?

    Motivation:

    You need to quickly capture and analyze requirements for building a prototype or proof of concept for an enterprise system.

    Suggestion:
    1. Capture any artifacts related to a business need. They may be some emails, presentation slides, a proposal document, a legacy system, some similar systems, a feature list or an initial requirement specification.
    2. Define terminologies (terms), build a background and context around the terminologies, expand the terminologies into business roles, business workflows, business components, business entities (objects), and business artifacts as much as possible.
    3. Identify the main stakeholders (roles) who will interact with the system (e.g. Guest, Admin, Developer, User, etc.) and their corresponding business problems or needs (i.e. their motivation of using the software system), and their current business workflows. Create as many scenarios (real world or domain business workflows) as possible. Create a domain model (a model of real world or domain entities, their properties and their relationships) if necessary.
    4. If there is an existing or similar system then identify the core components that each stakeholder will interact with (e.g. Accounts, Profiles, Reporting, APIs, Lessons, Videos, etc.) and their corresponding purposes.
    5. Identify the main tasks that each stakeholder will perform (e.g. Register an Account, Log in System, Create/Edit Profile, Create a Bucket, Generate an API Key, Create/Edit/Delete a Lesson, Create/Edit/Delete a Video, View Bandwidth, etc.).
    6. Identify the inputs and outputs and create mock-ups (sketches) or capture similar screens for each task. The quickest way to create a mock-up is to find similar existing screens in your system or external systems, and modify them. You can also search for templates of similar features in the Internet, and modify them. You can also brainstorm a  new one if you are good at creativity and imagining.
    7. Build or draw a prototype demonstrating 2 or 3 critical end-to-end workflows (An end-to-end workflow is a sequence of tasks that solves a real world problem completely). Contrast a current business workflows with the end-to-end workflows demonstrated by the prototype. The quickest way to create a prototype is to find similar existing workflow of of your system or external systems, and modify them. You can also repurpose a business workflow for a prototype. You can also brainstorm a  new one if you are good at creativity and imagining.
    8. Implement a proof of concept related to the 2 or 3 critical end-to-end workflows.
    9. For a complicated system, you may need to implement a proof of concept, build a prototype, and create a work break down structure or product backlog in parallel to ensure that the requirements are technologically and economically feasible.

     

     

    How to Quickly and Confidently Read Code?

    Motivation:

    You need to read existing source code to add a new feature or fix a bug or clone its features to another language or system.

    Suggestion:
    1. Try building and running the code without any modification.
    2. Try building and running the code with your inputs.
    3. Try changing a variable name to meaningful name and make sure that the change does not break anything.
    4. Try changing a package or module name to a meaningful name and make sure that that the change does not break anything.
    5. Try breaking a long method to smaller ones.
    6. Try breaking a large object or file to smaller ones.
    7. Try modifying or adding simple UI elements without touching the business logic or data layer.
    8. Try writing a unit test for a function.
    9. Try replacing an algorithm with with a better one.
    10. Try replacing an old component with a newer or better one.
    11. Try recreating the code structure from the separate components. If you cannot do this then try to understand the architecture of the existing source code.
    12. If you get an error when making the changes above then try showing the error on client side.
    13. If you cannot show the error on client side then try identifying the logic flow of the code related to the error, debug information, and learn about new terminologies or concepts in the code.

    Example:

    const maskPhone = (val) => {
        const x = val.replace(/\D+/g, '').match(/(\d{0,3})(\d{0,3})(\d{0,4})/);
        return !x[2] ? x[1] : `(${x[1]}) ${x[2]}` + (x[3] ? `-${x[3]}` : ``);
    };
    export { maskPhone }

    If you are not sure about what the JavaScript code above does then try to learn

    Another example:

    <div class="container">
        <iframe class="responsive-iframe" src="https://www.youtube.com/embed/u9Dg-g7t2l4"></iframe>
    </div>
    .container {  
        position: relative;  
        overflow: hidden;  
        width: 100%;  
        padding-top: 56.25%; /* 16:9 Aspect Ratio (divide 9 by 16 = 0.5625) */
    }
    /* Style the iframe to fit in the container div with full height and width */
    .responsive-iframe {  
        position: absolute;  
        top: 0;  
        left: 0;  
        bottom: 0;  
        right: 0;  
        width: 100%;  
        height: 100%;
    }
    

    If you are not sure about what the HTML and CSS code above  do then try to learn

    • how many ways an element can be placed in a web page (a document),
    • what is the position relationship between an element and its parent,
    • what is the position relationship between an element and its siblings,
    • what is the position relationship between an element and the document root, and
    • what position: relative , position: absolute , overflow: hidden do.

     

     

    How to Use Git

    Motivation:

    You want to use Git to version your files or share your files with other people.

    Solution:
    • Register a GitHub or a GitLab account.
    • Create a GitHub repository or a GitLab project.
    • Download and install a Git client.
    • Generate a personal access token.
    • Pull (checkout) a remote repository (e.g. https://github.com/huybien/asp.net-core.git) to an empty local folder (e.g. C:\Users\admin\Downloads\code).
      git init
      git config user.email "[email protected]"
      git config user.email
      git config user.name "Huy Bien"
      git config user.name
      git config credential.helper ""
      cd C:\Users\admin\Downloads\code
      git remote add origin -f https://github.com/huybien/asp.net-core.git
      git pull origin main
      / * or * /
      git pull origin master
    • Pull (checkout) a remote repository (e.g. https://github.com/huybien/asp.net-core.git) to a local folder that contains existing code (e.g. C:\Users\admin\Downloads\code).
      cd  C:\Users\admin\Downloads\code
      git init --initial-branch=main
      git config user.email "[email protected]" git config user.email git config user.name "Huy Bien" git config user.name
      git config credential.helper ""
      git remote add origin https://github.com/huybien/asp.net-core.git
      git fetch --all
      git add *.*
      git commit -m "new files added"
      git push -u origin main
    • Push local files to a remote empty repository.
      git init
      git config user.email "[email protected]"
      git config user.email
      git config user.name "Huy Bien"
      git config user.name
      git config credential.helper ""
      git add *.*
      git commit -m "first commit"
      git branch -M main
      git remote add origin https://github.com/huybien/asp.net-core.git
      git push -u origin main
    • Push changes to a remote repository.
      git config user.email "[email protected]"
      git config user.email
      git add *.*
      git commit -m "CP form"
      git branch -M main
      git push -u origin main
    • Update (fetch and merge) a local repository.
      git pull origin main
      /* or */
      git branch --set-upstream-to=origin/main main
      git pull
    • Force updating (fetch and overwrite) the current repository.
      git fetch --all
      git reset --hard origin/main
      git clean -fd git pull
    • Force updating (fetch and overwrite) a local repository (e.g. C:\Users\admin\Downloads\code).
      git -C C:\Users\admin\Downloads\code fetch --all 
      git -C C:\Users\admin\Downloads\code reset --hard origin/main 
      git -C C:\Users\admin\Downloads\code clean -fd
      git -C C:\Users\admin\Downloads\code pull
    • Reset (Revert) a local repository to a previous version.
      cd C:\Users\admin\Downloads\code
      git log --oneline
      git reset --hard 4355842
      // where 4355842 is a version id.
    • Remove all cached files.
      git rm -r --cached .
    • Display remote URL.
      git config --get remote.origin.url

       

     

     

    How to Quickly and Reliably Fix a Bug

    Problem:

    You need to fix a bug. However it takes you a lot of effort to fix. The fix may also not be reliable. How can you avoid this situation?

    Solution:
    1. Try to understand the scenario or use case. Ensure that you and the tester are talking about the same thing and agree about the missing or incorrect elements by comparing the final user interface with the original approved use case or user story or specification.
    2. Try to reliably reproduce the bug. It is okay if this attempt may not be successful.
    3. Try isolating the bug in the same or similar environment by using specific, smaller data and fewer settings. It is okay if this attempt may not be done due to your inherent complex software.
    4. Search for an existing solution using error message generated by the system. Include any library or framework name and version, and operating system name and version in search key words. If there is already an existing solution then this attempt can save us a lot of effort.
    5. Try to understand all the concepts in error message.
    6. Debug and log messages to identify the exact location in the source code that causes the issue. In order to to this we need to do the followings.
      • Identifying the flow of the data, i.e. the use case, the entry point and the exit point in the code related to the issue.
      • Trying to understand programming language syntax in the code. Do not guess anything.
      • Trying to understand purpose, inputs and outputs of library functions related to the use case. Again, do not guess anything.
      • Trying to understand data structure and a part of the database schema related to the use case.
      • Trying to review some concrete values inside the database if possible.
      • Trying to understand concepts, algorithms and architecture related to the use case. Again, do not guess anything.
      • These steps may be done in parallel and iteratively.
    7. Guess a cause of the problem based on the information that you can get in the sixth step.
    8. Try to isolate the issue, i.e. try to reproduce the issue using specific code and unit tests, if possible.
    9. Search for or propose a solution for the cause, i.e. propose a fix.
    10. Implement and test the fix.
    11. Repeat from step 5 to step 10 if necessary.

     

    Topic 11 – Software Construction

    Why do I need to learn about software construction?

    Knowing how to write code does not mean that you know how to create real-world software.

    In real world projects, you will need to know how to manage your code, how to read the existing code, how to write code following standard styles, how to ensure that your code is working, how to automate your the process of building, testing and deploying your code, how to handle errors in your application, how to optimize your code for better speed, how to write secure code, how to avoid code duplication, how to create readable code, how to create code faster.

    That’s why you need to learn about software construction.

    What can I do after finishing learning software construction?

    You will know how to create real world software with a team.

    Hmm! Is it really useful?

    If you doubt its usefulness then you can delay learning it until you are tasked to create a software system and you complete a half of it and are stuck there because when you add one more feature you will get tons of bugs due to the new code, or when you finish fixing 1 bug, you get 3 other bugs due to the code modification.

    An other scenario is when it takes another person 2 weeks to read 1000 lines of the code that you wrote in order to fix a bug or to add a new feature because your code is unstructured.

    Another scenario is when you are asked to improve some existing code for better performance or to refactor some code for clarity before adding a new feature but you do not know how to accomplish the task.

    Alright! What should I do now?

    Software construction requires a lot of reading. In order to get familiar with software construction you will need to read at least below books. Please read this Steve McConnell (2004). Code Complete. Microsoft Press book first.

    After that please read this Jon Loeliger and Matthew McCullough (2012). Version Control with Git: Powerful Tools and Techniques for Collaborative Software Development. O’Reilly Media book.

    Alternatively, you can read this Ben Collins-Sussman et al. (2011). Version Control with Subversion book.

    After that please read this Paul M. Duvall et al. (2007). Continuous Integration Improving Software Quality and Reducing Risk. Addison-Wesley book.

    After that please read this Robert C. Martin (2009). Clean Code: A Handbook of Agile Software Craftsmanship. Pearson Education book.

    After that please read
    – this Andy Hunt, Dave Thomas and Matt Hargett (2007). Pragmatic Unit Testing in C# with NUnit. Pragmatic Bookshelf book, and
    – this Kent Beck (2002). Test Driven Development: By Example. Addison Wesley book.

    After that please read
    – this Martin Fowler et al. (1999). Refactoring Improving The Design Of Existing Code. Addison Wesley book, and
    – its newer version Martin Fowler (2019). Refactoring. Improving the Design of Existing Code. 2nd Edition. Addison-Wesley Professional also.

    If you have to work with legacy code then please read
    – this Michael Feathers (2004). Working Effectively with Legacy Code. Prentice Hall PTR book, and
    – this Diomidis Spinellis et al. (2003). Code Reading: The Open Source Perspective. Addison-Wesley Professional book.

    After that if you are familiar with .NET then please read
    – this Matthew MacDonald and Bill Hamilton (2003). ADO.NET in a Nutshell. O’Reilly Media book to learn how to use a data provider to access data sources, and
    – this Brian L. Gorman (2020). Practical Entity Framework. Apress book to learn how to use an ORM to access data sources, and
    – this Suhas Chatekar (2015). Learning NHibernate 4. Packt Publishing book to learn how to use an ORM with mapping technique to access data sources, and
    – this Brian Larson (2017). Microsoft SQL Server 2016 Reporting Services. McGraw-Hill Education book to learn how to use a reporting system, and
    – this Justin Richer and Antonio Sanso (2017). OAuth 2 in Action. Manning Publications book to learn how to use authentication and authorization frameworks and libraries, and
    – this Matt Perdeck (2010). ASP.NET Site Performance Secrets. Packt Publishing book to learn how to reduce a web application response time.

    After that please read
    – this Elton Stoneman (2020). Learn Docker in a Month of Lunches. Manning Publications book, and
    – this Elton Stoneman (2021). Learn Kubernetes in a Month of Lunches. Manning Publications book. You should learn basic concepts, then try exploring the tool using official documentation if you get any issue with example code in the book.

    After that please read this Yevgeniy Brikman (2022). Terraform – Up and Running. O’Reilly Media book.

    Terminology Review:

    • Version Control.
    • Coding Standards.
    • Unit Tests.
    • Continuous Integration.
    • Refactoring.
    • Legacy Code.
    • Code Reading.
    • Data Providers.
    • Object/Relational Mapping (ORM).
    • Reporting Services.
    • Authentication.
    • Authorization.
    • OAuth 2.
    • Proof Key for Code Exchange (PKCE).
    • JSON Web Token (JWT).
    • OpenID Connect (OIDC).
    • The Backend for Frontend Pattern.
    • Performance.
    • Containers.
    • Container Images.
    • Container Image Layers.
    • Docker.
    • Docker CLI.
    • Dockerfiles.
    • Docker Volumes.
    • Bind Mounts.
    • Docker Compose.
    • Container Orchestrator.
    • Swarm.
    • Kubernetes.
    • Deployments.
    • ReplicaSets.
    • Pods.
    • Deployment Manifest.
    • ClusterIP Services.
    • LoadBalancer Services.
    • NodePort Services.
    • ExternalName Services.
    • headless Services.
    • Namespaces.

    After finishing learning about software construction please click Topic 12 – Software Testing to continue.