Software Engineering Research

Welcome to the field of software engineering research!

You may have been developing software for years and recently realized that you love it. However, you now want to go beyond completing routine tasks with conventional methods and tools.

You’re eager to improve how you tackle more challenging tasks—wanting to do them faster and at a lower cost. You aim to create innovative methods and tools that help others systematically develop software. If this resonates with you, then software engineering research is the perfect path for you.

1. Background preparation

Please read

Although these books offer a general overview and a broad perspective of software engineering, they do not cover every detail of the topics discussed. Instead, they provide summaries of key problems in software engineering and the most widely accepted solutions to those problems.

In other words, these books lay the foundation for you to begin conducting software engineering research.

While reading these books you can also review all the topics on this website for detailed information of each topic these books mention.

After that please read this Andy Oram and Greg Wilson (Editors) (2010). Making Software. What Really Works, and Why We Believe It. O’Reilly Media book to familiarize yourself with the perspective of software engineering researchers.

2. Software engineering research methods

Please read this William M.K. Trochim (2006). Research Methods Knowledge Base book to get familiar with research terminologies, types, and methods first.

To begin conducting software engineering research, you need to understand what can be investigated, how to conduct the investigation, what outcomes will be produced, and how to ensure that your findings are both correct and useful.

It means that you should clarify

  • what is your research subject (Is it a laboratory experiment or a real world project?),
  • what is your unit of analysis (Is it the whole or a part of the research subject? Is it a a software project or a requirement process or a design process or a testing process or a group of people or a model or a notation or a data structure or an algorithm?),
  • what is your research  result or research output or research outcome (Is it a model or a method or a technique or a tool or a notation or an answer or an judgment?)
  • what you will do to your research subject and how you can obtain your research result (How do you collect and analyze data? How do you formalize or model or represent the problem? How do you formalize or model or present your solution?),
  • which method is used to validate your result (Is it a mathematical proof or a controlled experiment or a case study?)

Please read the following papers or books to gain knowledge about research subjects, research results, data collection and analysis, and research validation.

Then, please read the following papers or books to gain knowledge about research subjects, research results, data collection and analysis, and research validation, specifically applied to the software engineering research field.

3. Research problem identification

This is the most challenging task. Before identifying a research problem, you should carefully review your background knowledge and interests. Once you have identified a potential research problem, you should revisit and double-check the prerequisite knowledge required to address the problem, ensuring that you are capable of undertaking the research you’ve chosen.

    3.1. Asking for a research problem

    The easiest and fastest way to identify a research problem is to consult with your advisor. They may already have a funded research project with various problems that need to be addressed.

    3.2. Searching for a research problem

    If you do not have an advisor, or if your advisor does not have a funded research project for you to work on, you can visit some of the websites listed below to explore concrete research topics that interest you. While doing so, try to refer to your background knowledge and experiences whenever possible.

    Hopefully, after browsing these websites, you will be able to identify some keywords that interest you, such as software architecture, microservice architecture, or multi-tenancy architecture.

    Example research topics might be:

    • Multi-tenant event-driven microservice architecture
    • Storage models and algorithms in multi-tenant environments
    • Empirical study on the impact of AI-assisted development tools on software development productivity and quality
    • Automated deployment of cloud-based enterprise systems

    After a research topic is identified and selected with some concrete keywords, there are 2 approaches you can take to find a research problem: (i) doing a literature review or (ii) searching for tool or method limitations.

    3.2.1. Doing a literature review

    The core idea of this approach is to search for a concrete problem from a general problem that is discussed in research papers.

    Please select a recent paper that interests you and read it carefully. Then, analyze the paper to answer the following questions:

    • What is the problem that the paper tries to address?
    • To which real world software systems has the solution proposed by the paper been applied? It is fine that the answer for this question may not be available.
    • What are the other primary approaches that have been proposed to solve the same problem posed by the paper?
    • How can these solutions be compared to one another?
    • Are there any data and source code illustrating an implementation of the solution that may be used for solution verification and validation?
    •  What are the remaining issues that the paper has not completely solved?
    • Are there any directions for improving the solution? Why? It is fine that the answer for these questions may not be available.

      If the selected paper you have just read only poses a general problem (an indicator of this situation is that you cannot answer most of the questions above), then the paper you’re interested in may be a secondary study. In this case, you will need to find a primary study that addresses a more concrete problem. You can use the websites below to search for a primary study by using some of the keywords found in the secondary study paper:

      Alternatively, you can select another paper that interests you, read and analyze it, and try to answer the questions to identify a research problem. You can repeat this process until you find a concrete problem for your research.

      3.2.2. Searching for tool or method limitations

      The core idea of this approach is to examine a software system that you are currently involved in.

      First, identify the tools (such as libraries, frameworks, external systems, architectures, algorithms, or notations) and methods related to your selected research topic that are being used to develop the software system’s features.

      Then, try to identify which parts of these tools and methods are still imperfect and need improvement, or what is missing in these tools and methods that makes them difficult or unable to fulfill specific requirements.

      If you follow this approach, please note the following:

      • Research results are the improved tools or methods, NOT the software systems that you are developing. These results need to be applied to your software system, and potentially to other systems as well, to evaluate their effectiveness or applicability.
      • If the software system that you are involved is trivial or the tasks can be done using the existing solutions without any issues, there may be no business need of improving the tools and methods. Therefore, providing context and motivation examples is critical when identifying a relevant problem.
      • Ensure that the results you propose have not been produced before, or that these results have not been published for the community. Your results should bring benefits not only to yourself but also to the wider community. This should be clearly stated in the context and motivation examples.
      • Be mindful of intellectual property issues. Most reviewers of your research results may need access to the source code of your software system to evaluate your contribution.

        4. Solution investigation

        Hopefully, at this point you will have spotted an unsolved problem that you are interested in.

        Then you shall need to iteratively

        • Identify research questions related to the problem.
        • Build your background knowledge related to the problem and research questions.
        • Check the literature for existing solutions.
        • Search for tools and methods related to the problem and research questions.
        • Propose your solution to the problem (i.e. providing your answers to the research questions). You may extend an existing tool or method to address some of its disadvantages. You may build a novel tool or method to solve a problem.
        • Document your solution.

        A note is that if you conduct an applied research, or a validation research, or an evaluation research, or an experience report then you may skip investigating a solution. Instead you leverage existing solutions.

        5. Solution evaluation

        • You may apply an existing tool or method to a system in a specific context, and collect evidences to prove that the problem can be solved using the tool or method. This approach may be used for applied research, validation research, evaluation research, or experience report. For this approach, you need to ensure that this study has never been conducted by someone else in a similar context, and your result does bring benefits to community.
        • You may implement a software system to help others use an existing tool or method more effectively. This approach can be used for applied research. For this approach, you need to clearly clarify the benefits when applying the tool or method by using your software system.
        • You may apply two existing tools or methods to develop the same software system, and collect evidences (e.g. performance or scalability) to help others to select the better one in a specific context. This approach can be used for applied research.
        • You may soberly and critically evaluate the effectiveness (i.e. the cost, the time, the fix) of your solution by comparing it with existing solutions or baselines using benchmark inputs or prove the correctness of your solution using mathematical modeling and logical reasoning.
        • Regardless the selected approach,  you always need to clearly clarify a real business need of your research result.

            6. Notes

            There are three core stages of a complete research process: discovering a problem, investigating a solution, and evaluating the solution.

            However, not all research needs to be fully complete. Some research efforts may focus on just one of these stages—such as problem identification, solution investigation, or solution evaluation—or on combinations of these stages.

            A research method, e.g. epxeriement or case study or survey, may be applied to one, two, or all three stages of a research process. Different research methods may be used for different stages of the same research project.

            A research method, such as an experiment, case study, or survey, may be applied to one, two, or all three stages of the research process. Different research methods can be used for different stages within the same research project.

            Personally, I believe that for research results to be truly innovative, at least one of the three stages used for conducting the research must involve a trial-and-error approach, rather than following a prescriptive process model.

            7. Writing your paper

            Hope that you will have found a research problem, solved it, finished documenting and evaluating your solution. Then you will need to write your paper. Your paper must

            • Clearly articulate the topic of research,
            • Summarize the critical problems and current solutions used in the field,
            • Describe the proposed research concepts that constitute your solution, the intended methodology, the evaluation strategy, and results to date, and
            • Identify the contributions of the proposed work.

            You can use the links below to check the grammar and fluency of your paper if you are not a native English speaker:

            You can use the Office2007_BibWord tool to manage references in your paper if you plan to write it using MS Word 2007 or MS Word 2010.

            8. Publishing your paper

            After that, you will want to choose a conference and/or publisher to submit your paper to. Please use the links below for this purpose.

            Good luck with your research journey!

             

            (Visited 1,567 times, 1 visits today)

            Leave a Reply

            Software development and software engineering research