Dynamics 365 Business Central Outage February 7, 2023 UTC

Dynamics 365 Business Central

Hi, Readers.
Dynamics 365 Business Central SaaS was down again early this morning Japan time.
In this post, I would like to briefly share what I encountered and some information from Microsoft. Hope to help you. (I will update if there is an update from MS)

Here is the information from the Business Central Community and Twitter.
Is D365 BC Down?
My twitter

Update: On February 9 at around 6 pm Japan time, once again the same situation as yesterday occurred, and this time it was global, automatically recovering after about 15 minutes.

More info:

Update: At 5 pm Japan time, some tenants have been fixed.

Service Restored 2023/02/08 (Wed) 18:17

First I tried opening the Sandbox environment, but the following error was prompted.

We could not find the page you were looking for.

TECHNICAL DETAILS
An unknown error occurred. If you continue to see this error, please provide the following information when requesting support.

Then I try to open the Business Central Admin Center, after a very long wait, the following error was prompted.

Operation timed out
The requested operation timed out

Update: Microsoft updated the error message……

Sorry, Business Central is temporarily unavailable. Please try again later.
For more details, reach out to your system administrator.

TECHNICAL DETAILS
System administrators can find more details on the Service Health Dashboard

I thought it might be a problem with the trial tenant, so I went to the paid tenant and tried it, and the situation was the same.

What about the production environment?

If the BC URL you use is Tenant ID + Environment Name, it will always stop in the screen below.
For example,

https://businesscentral.dynamics.com/d8f36038-1f93-4543-affc-5dc92b6ee871/production

Sometimes the following page will pop up.😑

If you log in from the BC homepage, you can open the production environment.

Access your Dynamics 365 Business Central account

The Tenant ID is not included in the URL at this time.

https://businesscentral.dynamics.com/?RedirectedFromSignup=1

Or try the BC URL directly:
https://businesscentral.dynamics.com/

So recalling the Sandbox and Admin Center issues above, as long as the URL has a Tenant ID (Domain), it cannot be accessed. So I suspect that if you have multiple production environments in your tenant, you may not be able to open either.

Updated: It seems that it has nothing to do with the Tenant ID, as long as the environment name is added, the problem will occur.
For example,
OK:

https://businesscentral.dynamics.com/d8f36038-1f93-4543-affc-5dc92b6ee871/

NG: According to feedback, in some environments, the environment name will be automatically added when accessing BC. In this case, you can only wait for Microsoft to fix it.

https://businesscentral.dynamics.com/d8f36038-1f93-4543-affc-5dc92b6ee871/Production

PS:
1. Although the Sandbox environment cannot be opened, the extension can still be published from VS Code.😑

2. Although the production environment can be opened, extensions cannot be installed.
PTE:

An unexpected http error has occurred. HttpErrorCode: InternalServerError

AppSource Extension:

Okay, here’s the information from Microsoft.

You can view the health of your Microsoft services, including Office on the web, Yammer, Microsoft Dynamics 365, and mobile device management cloud services, on the Service health page in the Microsoft 365 admin center.

Go to the Microsoft 365 admin center at https://admin.microsoft.com, and sign in with an admin account.

In the admin center, go to Health > Service health.

Dynamics 365 Business Central Service health:

February 8, 2023 6:17 PM

Service Restored

February 8, 2023 5:00 PM

Title: Admin center unavailable

User impact: Admins may be unable to access your Business Central admin center.

More Info: Environment access is unaffected for customers outside SE Asia region.

Current Status: We completed development and deployment of the hotfix. Following deployment of the hotfix, we identified additional impact within Business Central admin center. We have developed a secondary hotfix which is in process of being deployed. We are monitoring service diagnostics for successful deployment of the fix.

Incident Start Time: Tuesday, February 7, 2023 at 8:37 PM UTC

Next Update: Wednesday, February 8, 2023 at 12:00 PM UTC

February 8, 2023 11:46 AM

Title: Admin center unavailable

User impact: Users may be unable to access your Business Central admin center

More Info: Environment access is unaffected for customers outside SE Asia region.

Current Status: We are developing a hotfix which will change Business Central admin center pathing to a healthy infrastructure within the upstream service.

Incident Start Time: Tuesday, February 7, 2023 at 8:37 PM UTC

Next Update: Wednesday, February 8, 2023 at 8:00 AM UTC

To allow time for hotfix development and deployment.

February 8, 2023 9:09 AM

Title: Admin center unavailable

User impact: Users may be unable to access your Business Central admin center

More Info: Environment access is unaffected.

Current Status: After our investigation, we determined that an outage for an upstream service is causing all attempts to access the admin center to receive a timeout error. We are investigating service diagnostics to determine next steps.

Incident Start Time: Tuesday, February 7, 2023 at 8:37 PM UTC

Next Update: Wednesday, February 8, 2023 at 3:00 AM UTC

February 8, 2023 8:52 AM

Title: Admin center unavailable

User impact: Users may be unable to access your Business Central admin center

We are aware of an emerging issue where users are experiencing an inability to access your Business Central admin center. We are investigating the issue and will provide another update within the next 30 minutes.

Microsoft 365 Service health:

February 8, 2023 11:51 AM

Title: Users may be unable to use various Microsoft 365 apps or features

User impact: Users may be unable to use various Microsoft 365 apps or features.

More info: This message is related to SHD post TM512596. We’ve determined that various additional Microsoft 365 services are affected as part of the overall event. We will continue updating this list as we learn more about this issue: -Microsoft PowerApps: Users may be unable to access the service -Microsoft Intune: The data syncing issue from Azure AD in Microsoft Intune has been remediated, but there is additional impact in the Asia Pacific region. Affected users may intermittently be unable to enroll and check in devices to Microsoft Intune -Microsoft 365 Defender: Users may notice stale data for Advanced Hunting, Insider Risk, and Privacy Management Microsoft Purview: Admins are unable to manage cases and start collections with Microsoft Purview eDiscovery (Premium) -Universal Print: Users may be unable to print -Windows 365 Cloud PC: Users may be unable to connect to Windows 365 Cloud PC or provision Cloud PCs in the affected region

Current status: We have rerouted the affected traffic for a majority of the impacted services. In parallel, we are continuing to monitor the service and addressing any residual degradation associated with this event to ensure the issue is fully resolved. We understand the impact that this issue may have on your organization and we are dedicated to urgently addressing the service degradation associated with this issue.

Scope of impact: This issue primarily impacts users hosted through the Asia Pacific and Australia regions but could impact any user if served through those regions as well.

Start time: Wednesday, February 8, 2023, 5:25 AM (2/7/2023, 8:25 PM UTC)

Root cause: A portion of infrastructure supporting various Microsoft 365 apps is operating below acceptable thresholds.

Next update by: Wednesday, February 8, 2023, 1:00 PM (4:00 AM UTC)

February 8, 2023 10:38 AM

Title: Users may be unable to use various Microsoft 365 apps or features

User impact: Users may be unable to use various Microsoft 365 apps or features.

More info: This message is related to SHD post TM512596. We’ve determined that various additional Microsoft 365 services are affected as part of the overall event. We will continue updating this list as we learn more about this issue: -Microsoft PowerApps: Users may be unable to access the service -Microsoft Intune: Users can’t sync data from Azure Active Directory (Azure AD) in Microsoft Intune.  

Current status: We’ve finishing our efforts to reroute the affected service traffic to alternate infrastructure to mitigate the issue. We’ll continue monitoring the service health and perform any additional steps necessary to fully resolve the issue for all users.

Scope of impact: Based on our investigation, this issue appears to be primarily impacting users located or hosted in the Asia Pacific region. 

Start time: Wednesday, February 8, 2023, 5:25 AM (2/7/2023, 8:25 PM UTC)

Root cause: A portion of infrastructure supporting various Microsoft 365 apps is operating below acceptable thresholds. 

Next update by: Wednesday, February 8, 2023, 12:00 PM (3:00 AM UTC)

February 8, 2023 9:24 AM

Title: Users may be unable to use various Microsoft 365 apps or features

User impact: Users may be unable to use various Microsoft 365 apps or features.

More info: This message is related to SHD post TM512596. We’ve determined that various additional Microsoft 365 services are affected as part of the overall event. We will continue updating this list as we learn more about this issue: -Microsoft PowerApps: Users may be unable to access the service -Microsoft Intune: Users can’t sync data from Azure Active Directory (Azure AD) in Microsoft Intune.  

Current status: We’ve identified a specific portion of our hosting infrastructure that is performing below acceptable levels and are working to move service to alternate, healthy infrastructure. In parallel, we’re evaluating alternative mitigation options to restore service as quickly as possible. 

Scope of impact: Based on our investigation, this issue appears to be primarily impacting users located or hosted in the Asia Pacific region. 

Start time: Wednesday, February 8, 2023, 5:25 AM (2/7/2023, 8:25 PM UTC)

Root cause: A portion of infrastructure supporting various Microsoft 365 apps is operating below acceptable thresholds. 

Next update by: Wednesday, February 8, 2023, 11:00 AM (2:00 AM UTC)

Azure status:

Datacenter Cooling Event – Southeast Asia
Impact statement: Starting around 20:19 UTC on 7 February 2023, a utility power surge in the Southeast Asia region tripped a subset of the cooling units for one datacenter offline. While working to restore the cooler units, temperatures in the datacenter increased so we have proactively powered down a small subset of selected compute and storage scale units, to avoid damage to hardware and reduce cooling system load. All impacted storage and compute scale units are in the same datacenter, within one of the region’s three Availability Zones (AZs). Multiple downstream services have been identified as impacted.

PS:
1. Dynamics 365 Business Central: View the health status and subscribe to notifications

2. Dynamics 365 Business Central: Where to file bugs ,issues and Questions

END

Hope this will help.

Thanks for reading.

ZHU

コメント

Copied title and URL