Category: Experiential

During my first week of internship, Mr. Peter assigned me several tasks to enhance my understanding of software development tools and technologies. On the first day, I was introduced to Bitbucket, a software development platform used for storing, tracking, and collaborating on software projects. We practiced creating merge conflicts and resolving them, which helped me gain a deeper understanding of version control systems.

For the rest of the day, Mr. Peter gave me four tasks to focus on. First was TypeScript, a syntactic superset of JavaScript that adds static typing. After learning this programming language, I feel that TypeScript is beneficial as it makes code easier to maintain and reduces the occurrence of bugs. Next, I learned React fundamentals, understanding that React is a JavaScript library that allows developers to create user interfaces. Third was Next.js, an open-source web development framework that provides React-based applications with server-side rendering capabilities. I’ve found it to be an excellent web development framework because it saves users’ storage and provides a smooth web browsing experience. Lastly, I started learning about React testing using Jest, though I’m still in the learning process and don’t have extensive knowledge to share about this topic yet.

In conclusion ,this week has provided me with valuable insights into version control and frontend development technologies, setting a strong foundation for my internship.

Monday, October 14, 2024 – At the start of the previous week, Mr. Peter introduced me to our upcoming project, which will use Next.js for the website and ASP.NET for the backend API. However, instead of using EF Core, we’ll be adopting Dapper for mapping database queries. To help me get up to speed with Next.js, Mr. Peter provided access to a course and other resources to familiarize myself with the technologies involved in this project.

In regard to last week’s issues with message queue consumers in RabbitMQ, Mr. Peter addressed all of my concerns and provided clear explanations. I was very grateful for his guidance in correcting my misconceptions, especially on the Dead Letter Exchange (DLX). With the RabbitMQ issues resolved, I shifted my focus back to learning Next.js.

After completing the Next.js course, I immediately started applying what I had learned. Since we’re using TypeScript in our project, I initially struggled a bit with ensuring that the code I wrote adhered to clean code principles, especially avoiding duplication. Next.js is a powerful React framework that simplifies both server-side rendering and client-side rendering. To create a user interface, I applied Tailwind CSS, which made styling much easier and more efficient.

For data display, I created a dummy JSON dataset and used an online free API that provides images to enhance the product list. This allowed me to develop a simple page where a list of products is shown, and each product can be clicked to view its details on a separate page.

Towards the end of the week, I successfully built a basic dummy page that simulates a product listing with clickable details. Mr. Peter also explained the overall project flow and the different entities we will be working on in the coming week. This project will incorporate multiple databases, including PostgreSQL, MongoDB, and AWS, each chosen based on their strengths for specific tasks. I’m looking forward to diving into this exciting project of multi-database architecture as we move forward.

Monday, October 7, 2024 – Last week, I integrated a new logic for updating bulk modified entities from one context into another within an existing function. Initially, everything worked fine. However, during further testing, I discovered a loophole in the implementation. If only the name was changed, the update worked as expected. But when the user also changed the associated data code (which was used as the search key), the system created a new data entry instead of updating the existing one. After discussing this with Mr. Peter, we decided not to continue with this implementation, as it became unnecessary once the RabbitMQ message handling stabilized.

I then refocused on refining the message queue service. I encountered an issue when running the queue three times in a row: the system threw an error stating, “A second operation started on this context instance before a previous operation completed.” This was due to different threads trying to use the same instance of DbContext concurrently. To resolve this, I implemented a scope within the update data function, ensuring that each operation had its own context instance. Once completed, I tested it and it worked. However, a new issue emerged, the message execution was not happening in the correct order.

After some research, I found that implementing a semaphore could help enforce proper message execution order. Still unsure if this was the best solution, I consulted Mr. Peter. He pointed out that using a semaphore wasn’t necessary, as the issue likely stemmed from unrefined code elsewhere. The next day, Mr. Peter made minor adjustments to the code and explained that the unnecessary use of the using() function was limiting how the message queue was consumed.

With this issue resolved, I then revisited the message flow to identify potential errors or loopholes. I encountered another problem on the consumer side. If an error occurred during message processing, the consumer would issue a basic nack (negative acknowledgment), and the message would be discarded. To avoid data loss, I researched how to handle this more effectively. RabbitMQ’s documentation suggested using Dead Letter Exchange (DLX), a method that redirects failed messages to a dedicated queue for further inspection or retries.

By the end of the week, I was able to implement the DLX mechanism, so if a message received a basic nack instead of an ack, it would be sent to the DLX queue. However, a concern arose as instead of just one message being sent to the DLX queue, four messages were sent. This remains an issue to address.

Monday, September 30, 2024 – Last week, I continued the cleanup of old and unused features, focusing on the entity models that were no longer relevant. This process involved verifying that all removed features were not in use elsewhere in the code before I executed the migration and updated the database.

Previously, I had tried to create a separate connection service for RabbitMQ to avoid hardcoding the connection details. However, when testing this service, I found it failed to read the connection parameters. After consulting with Mr. Peter, I learned that the RabbitMQ connection service was unnecessary and could be simplified by replacing specific hostnames with a general hostname defined in the config file.

Next, while enhancing the message queue, I encountered an issue with publishing messages. I had initially implemented a “Publisher Confirm” mechanism, but it had drawbacks. Mr. Peter demonstrated an alternative approach that involved using transactions, along with commit and rollback procedures. This method helps ensure that messages are processed reliably, mitigating the risk of data loss.

As the message queue service is designed to update data from one context to another, it effectively processes updates one at a time after a message is successfully published. However, this raised a question: what happens to previously created data that has already undergone changes? One solution would be to delete and regenerate this data, but that would be highly inefficient, especially considering the processing time involved.

To address this, I initially considered creating a button to pull any modified changes from the original context. However, I later realized this was not the best approach. Instead, I decided to use the existing regenerate button to call out or update changes effectively. By the end of the week, I had successfully integrated this new logic into the existing function, although further testing is necessary to ensure that no errors or bugs will be introduced in the process.

Monday, September 23, 2024 – Last week, I focused on refining the controller tests, particularly in contexts that typically throw errors when executed all at once, but function correctly when run individually. The root cause was the unavoidable sharing of data within a single test file that contained both query and command API tests. Following Mr. Peter’s advice, I decided to separate the tests by query and command to eliminate this issue.

I also continued enhancing the message queue I was working on. I encountered a significant problem: when the RabbitMQ server is down from the start, starting the application caused a collapse. Initially, I thought this behavior was acceptable, but Mr. Peter pointed out that since RabbitMQ is implemented on the backend, it shouldn’t impact the user interface unless an API call triggers it. Upon further investigation, I found that several frontend services also utilized the message queue service, which contributed to the issue.

To address this, I extracted the used services into a new service that integrates the message queue functionality. By the end of the week, I presented my progress to Mr. Peter, and he noted that several aspects of the message queue service still needed enhancements, which I plan to tackle next week.

Monday, September 16, 2024 – Last week, I continued my work on completing the message queue. Although I had successfully consumed and received the messages, I was initially unable to execute the intended purpose of these messages. After presenting my attempts to Mr. Peter, I learned that the issue stemmed from an unnecessary transaction I had included. It turned out that the system only required a simple update function.

Once I ensured that this part of the message queue was functioning correctly, I proceeded to implement the message queue in various components that required it. This message queue facilitates communication between microservices, ensuring that relevant data is consistently updated across the system.

Mr. Peter tasked me with cleaning up several features and services that would no longer be needed in the future. My next challenge with the RabbitMQ implementation was determining the best way to handle errors. To test this, I stopped the RabbitMQ server and attempted to start the application, which caused it to shut down immediately. To mitigate this, I added a try-catch block inside the message service constructor to ensure that errors would be thrown rather than causing the application to crash. While this approach worked, I recognized that it was not a clean or best practice solution.

Determined to refine my approach, I created specific files for managing the RabbitMQ connection, aiming to avoid hardcoding and to handle errors more effectively. However, I encountered issues during testing; it appeared that I might have missed some initializations. I plan to revisit this in the coming week.

Monday, September 9, 2024 – Last week, while continuing to test the new button functionality, I encountered an error during the data cleanup process. The issue arose because the cleanup only read data from a single table, while the UI page displayed a combination of data from Table A and Table B. After identifying the problem, I created a new UI page to ensure the cleanup query also retrieves data from Table B, preventing discrepancies.

In addition to resolving the UI data issue, I needed to ensure the data cleanup logic was accurate and effective. Initially, the cleanup process was set up to automatically convert any null or zero values to zero. However, this approach did not fully address all potential data scenarios. After discussing the requirements with Mr. Peter, he provided insights into the different scenarios and the best approach for handling data cleanup, which refined my solution.

Next, I focused on the message queue implementation. I was tasked with creating a controller test to verify that the publish message functionality was correctly utilized. Although it took some time to understand the testing process for the message queue, Mr. Peter’s guidance helped me successfully create and execute the test. I then shifted my attention back to the consumer side, working on handling the execution after a message is received. I sought Mr. Peter’s advice to ensure the implementation was correct, and while there are still some adjustments and testing needed, I plan to continue with this next week.

Monday, September 2, 2024 – Last week, I discovered that the issue with not receiving messages was due to a failure in publishing them correctly. I identified and corrected the problem, particularly focusing on the message format in the publish function. After several adjustments, the message was successfully queued in RabbitMQ. I verified this by checking the RabbitMQ management UI, where I confirmed that the message had been published successfully. With the publishing issue resolved, I turned my attention back to the consumer side and made necessary fixes to ensure that messages are read correctly.

In addition, I was assigned a new task to clean up existing data, which had some inaccuracies in previous calculations. The cleanup process involves resetting the data and updating it with the correct values. I started by creating an API query in context A to retrieve the latest invoice and its corresponding data price. This latest data price is then used in a query from context B to update the data in context B.

Once both queries were completed, I moved on to the UI updates. I utilized an existing cleanup page to incorporate a new button feature, given its similar functionality to the current cleanup process. By the end of the week, most of the new button functionality was implemented, although I haven’t fully tested it yet. Testing will be a priority for next week.

Monday, August 26, 2024 – Last week, I continued my exploration and implementation of RabbitMQ message queues. Following Mr. Peter’s advice, I focused on encapsulating methods to be used by multiple components, a best practice that helps reduce duplication and simplifies error detection. It took several attempts to perfect the encapsulation for both the publisher and consumer before I could move on to actually using them to publish messages.

When setting up message publication, I faced a few challenges. I had to consider the type of messaging pattern: whether the message queue would be from a single publisher to a single consumer, or from a publisher to multiple consumers. For scenarios involving multiple consumers, the publisher needs to implement an ‘exchange’. An exchange in RabbitMQ is a routing mechanism that decides how messages should be distributed to queues based on routing rules. It essentially determines where messages should go. There are different types of exchanges, such as direct, fanout, topic, and headers, each with its own routing logic.

Next, I turned my attention to implementing the consumer side. One tricky part was figuring out how to call the Consume method effectively and ensure it returned the published message for further processing. To handle message consumption, I used a background service provided by Microsoft. After setting up both the consume and publish methods, I checked the RabbitMQ management UI and confirmed that the published messages appeared in the queue. However, the messages were not being consumed as expected. After revising the consumer setup, I consulted Mr. Peter and discovered that the issue was with the publisher failing to send the message properly. I’ll be tackling this issue in the coming week.

Monday, August 19, 2024 – Last week, I started by thoroughly testing the new feature for importing Excel spreadsheets, ensuring that it could handle various errors to provide a smooth user experience. Once I felt confident that most scenarios were covered, I deployed the latest changes to the cloud server.

Next, I shifted my focus to researching message queues, specifically RabbitMQ. I began by creating console projects following the provided documentation, but I encountered a roadblock when trying to run the Docker command to build the RabbitMQ container. I forwarded this issue to Mr. Peter, who guided me the next day on running the correct command, and thankfully, the Docker container was successfully built.

Simultaneously, I worked on developing a small feature that involved implementing a single button. Given the simplicity of the task, I prioritized completing it before returning to RabbitMQ testing. The following day, I successfully ran the feature but noticed that it lacked detailed error handling, and additional filters were needed. Moreover, the query associated with this feature required optimization, as it initially took about 7 seconds to execute. By the end of the week, I managed to optimize the query, reducing the execution time to just 1.5 seconds. However, I discovered a discrepancy in the calculated data from this feature compared to the existing data from another module.

Next week, my priority will be identifying and resolving this bug before I can proceed with the message queue task.