NL-4 - Serverius DC2 maintenance (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - NL-4 - Serverius DC2
  • Our datacenter partner Serverius is currently expanding NL-4 - DC2 with extra power and redundancy. As a result of this there is an upcoming maintenance window on the power feeds.

    We want to inform you of an upcoming maintenance:

    Location: NL-4 - Serverius DC2, Meppel

    Start: 08.03.2018 1:00 CET

    End: 08.03.2018 3:00 CET

    During this maintenance it may occour that your dedicated server will be power be switched off. All network services and other services (such as web / cloud- hosting) are not affected.   

     

  • Datum - 08-03-2018 01:00 - 31-12-2019 12:11
  • Laatste update - 07-03-2018 09:10
Dark fiber maintenances at NL-3, supplier (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - NL-3
  • You are receiving this message as an advance notice because our dark fiber suppliers “Relined” will do maintenances.

    Location = NL-3 - Zwolle, NL
    Service Type = Dark fiber

    Start: 2017-02-07 00:00:00 CET
    End: 2017-02-07 07:00:00 CET

    Start: 2017-02-08 00:00:00 CET
    End: 2017-02-08 07:00:00 CET

    Start: 2017-02-09 00:00:00 CET
    End: 2017-02-09 07:00:00 CET

    Reason: Precautionary maintenance.
    This work represents precautionary maintenance, Relined is obliged to carry out this work to maintain the quality of our fiber network.
    This activity will impact the SDC1AMS-NIKHEF and SDC1SDC2 fiber.

    Expected Outage: When during the maintenance window both fibers will be affected exactly at the same time, partial downtime to some internet routes could be possible. We asked Relined to do the maintenance fiber by fiber, they promised to try, but unfortunately no guarantee could be given.

    We apologize in advance for any inconvenience this might cause.

  • Datum - 07-02-2017 00:00 - 22-02-2017 14:04
  • Laatste update - 01-02-2017 13:31
Emergency maintenance NL-3 - Serverius DC1 (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - NL-3 - Serverius DC1 power path
  • #1 - 19:15 +1 GMT
    ---
    Our datacenter planned an emergency maintenance for the upcoming. All colo services will be effected by this. Please shutdown your server orderly prior to the maintenance and setup an automated boot within the BIOS to boot the server as soon as AC power is connected. See below for the full message provided by Serverius.

    -

    Like we communicated by our NOC website and our customers mailing list past night, we experience power instability at the A power divider. After long investigation by our power maintenance supplier Alewijnse the problem was found. One of the connectors of one of the main power phase copper bars (large copper cable) of the A divider is partly damaged. Therefore the main power is currently switched to also another power breaker and power switch. In the past day the engineers looked at all possible causes and also today our datacenter builder ICTroom calculated if all power values of all parts of the datacenter are still in range. A half hour ago they gave us green light, the results are all good. Therefore the next step will performed: maintenance to resolve the problem.

    Therefore we inform you with this email about emergency maintenance what will be performed upcoming night.

    Location = SDC1, Dronten, NL
    Service Type = Main divider switch and cabling replacement
    Maintenance window start: 29.07.2016 01:01 +1 GMT
    Expected power downtime duration: less than 1 hour

    Because the broken power bar is part of the main power path where the generator and normal power are connected it will be needed to power off the A power divider. Also because the power maintenance company is not allowed to work on live a high voltage systems. Therefore during this maintenance window all power of this divider will be disconnected. During this all colocation users what are using a single A power connection in their colocation rack will not have power. Users with an A and B power connection can just use the B feed because the B side will be unaffected.

    After disconnection the all power bars and the connected main power switch will be replaced for sure. This will prevent same possible problems in the future.

    Currently all preparation are made by the engineers. By the default safety procedure, all colocation users are from 19:00 hours +1GMT not allowed to enter the datacenter till the full maintenance is performed. The gate is closed. We kindly ask everyone not to visit the datacenter because the engineers are currently preparing and they need to do their work with clear focus and fast results.

    We will advise everyone to enable automatic booting on all your hardware and remotely shutdown your hardware before the maintenance window will start.

    After the maintenance is finished the broken parts will be investigated. In the upcoming week we expect to send all users the final conclusion and explanation as a clear “Reason For Outage” .pdf rapport.

    We are very sorry about this unexpected maintenance. Unfortunately we need to perform it in this short amount of time because Serverius cannot take any risk of broken part in our main power infrastructure because it could cause larger problems with devastating results.

    When we have news we let you know.

    -

    Sorry for any inconvience caused by this emergency maintenance.

  • Datum - 29-07-2016 01:00 - 29-07-2016 02:00
  • Laatste update - 09-10-2016 13:34
Packageloss (Opgelost)
  • Prioriteit - Hoog
  • Storingen Overige - CSW-02
  • #1 - 10:09
    We identified the origin and have been able to resolve the packageloss.

    #1 - 09:35
    Since this morning we are seeing an increased packageloss on our coreswitch CSW-02. We are working to resolve this issue asap.

     
    ---

    Sorry for any inconvenience caused.

  • Datum - 17-05-2016 08:35 - 17-05-2016 13:39
  • Laatste update - 17-05-2016 10:18
LVM write timeout (Opgelost)
  • Prioriteit - Gemiddeld
  • Storingen Overige - Cloud cluster
  • #1 - 18:40 -  28/02/2016 CET
    --- 
    At 18:03 we have experiened a time-out on one write cache SSD within our storage cluster. Due to this it is possible your server experieced a write time-out out on its LVM, causing your VM to mounted to read-only. As we are unaware of which VM's have been effected we kindly instruct you to restart your server within MyServeo or using SSH / terminal.
    ---

    We apologize for any inconvenience this may have caused.

    If there are any issues, please open a support ticket at support@serveo.nl.

  • Datum - 28-02-2016 18:03
  • Laatste update - 04-03-2016 16:13
VRRP failure #2 (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - CSW-02
  • #1 - 20:33 01/03/16
    --
    This evening we have seen a VRRP failure on both coreswitch uplinks at CSW-02 between 18:47 PM and 20:18. The cause of this issue has been resolved by our on-site and we are reviewing the steps taken in effort to prevent this in furture. 

    --

    Sorry for any inconvenience caused.

  • Datum - 01-03-2016 19:47 - 01-03-2016 20:18
  • Laatste update - 04-03-2016 16:13
VRRP failure (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - CSW-02
  • #1 - 11:41 12/02/16
    --
    This morning we have seen a VRRP failure on both coresiwtch uplinks at CSW-02 between 11:00 AM and 11:37. The cause of this issue has been resolved by our onsite and we have taken massures to prevent this from happening again. 

    --

    Sorry for any inconvenience caused.

  • Datum - 12-02-2016 11:00 - 14-02-2016 13:08
  • Laatste update - 12-02-2016 12:24
Defective switchport / cable (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - SW02
  • #2 - 04:32
    Our engineer has identified the defected cable and replaced it. All connectivity is recovered.


    #1 - 02:45
    At this moment we are facing an outage on a port of our dedicated server and shared hosting network due to a devective switchport / cable. Our engineers are driving to the datacenter to identify the issue and to resolve it.

    ---

    Sorry for any inconvenience caused.

  • Datum - 24-01-2016 02:42 - 24-01-2016 04:32
  • Laatste update - 07-02-2016 02:08
web06 automatische migratie fout (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Server - web06
  • #2 - 21:13
    Inmiddels zijn alle services weer beschikbaar onder normale load omstandigheden. Mocht u nog hinder ondervinden, neem dan gerust contact op via support@serveo.nl

    ---

    #1 - 20:01
    Door een fout tijdens opgedaan tijdens een automatische migratie kan het zijn dat u op dit moment hinder ondervind met de verbinding van uw website. Onze systeembeheerder werkt aan een oplossing om dit probleem zo spoedig mogelijk te verhelpen.

    Voor een actuele status verwijs ik u naar onze pagina https://my.serveo.nl/serverstatus.php.

    Onze excuses voor het ongemak.

  • Datum - 15-01-2016 20:01 - 15-01-2016 21:26
  • Laatste update - 15-01-2016 21:26
High platform loads (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - Cloud cluster
  • #2 - 16:50
    All loads are restoring back to normal levels. Due to a DDoS atack multiple servers suffered on high CPU usage and IO loads. We decided to speed up the implementation of our extra 10Gbe network and will implement this upcomming week.

    ---

    #1 - 14:40
    At this moment we are facing high loads troughout our cloud cluster. Our engineers are working to identify the cause and solve it.
    All cloud services and shared hosting services are effected by this issue.

    ---

    Sorry for any inconvenience caused.

  • Datum - 08-01-2016 14:40 - 08-01-2016 16:50
  • Laatste update - 13-01-2016 20:43
High platform loads (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - Cloud cluster
  • #2 - 14:43
    Currently all services have been recovered. As of a failure result on two compute resources (hypvisors), there has been an IO load caused to to auto migration / bootup. We are investigated this matter to prevent it happening again in the future.

    ---

    #1 - 13:00
    Currently we are facing a high load troughout the our cloud server platform. Our engineers are working to identify the cause and solve it.
    All cloud services and shared hosting services are effected by this issue.

    ---

    Sorry for any inconvenience caused.

  • Datum - 04-01-2016 13:00 - 04-01-2016 14:50
  • Laatste update - 06-01-2016 01:49
Migration / Migratie (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - AMS-1
  • #7 - 06:40 - 31/10/2015 CET
    ---

    De migratie is voldooid. Alle services van onze (cloud) omgeving zijn inmiddels weer online. De cloud servers worden in de wachtrij geboot om IO load te verminderen. De status hiervan kunt u terugvinden via my.serveo.nl onder de dienst pagina van uw cloud server.

    ===

    Migration complete. All (cloud) services are up and running. Currently we queed the bootup of cloud servers to minmize IO queuing. The status of your server can be found on my.serveo.nl at your cloud services page. 

    ---


    #6 - 04:50 - 31/10/2015 CET
    ---

    Inmiddels zijn alle dedicated servers correct geboot. Mocht je geen connectie kunnen maken met de server, neem dan contact op via support@serveo.nl.

    ===

    Meanwhile all dedicated servers are booted up correctly. If you have any connectivity issues, please contact us at support@serveo.nl.

    ---

    #5 - 03:15 - 31/10/2015 CET
    ---

    Alle colocated servers zijn gerackt en geboot in Dronten. Mocht je problemen ondervinden, neem dan contact op via support@serveo.nl.

    ===

    All colocated servers are racked and booted up in Dronten. If you have any issues, please contact us at support@serveo.nl

    ---

    #4 - 02:35 - 31/10/2015 CET
    ---

    Ons team is op locatie en zijn op dit moment de servers aan organiseren om in het rack te plaatsen.

    ===

    Our team is on spot and working to organize the servers to mount them in the racks.

    ---

    #3 - 23:47 - 30/10/2015 CET
    ---

    Op dit moment zijn alle powered off en is het geheel ingeladen voor vervoer naar Meppel en Dronten.

    ===

    Currently we are almost finished booting and load the servers for transportation to Meppel and Dronten.

    ---

    #2 - 18:40 - 30/10/2015 CET
    ---

    Zoals vorige week aangekondigd, voeren wij vanavond een data centrum migratie uit tussen 22:00 uur en 08:00 uur. Als u een dedicated, cloud of colocatie server heer, dan verzoeken wij u vriendelijk de server(s) voor 22:00 uur af te sluiten.

    Bezoek https://my.serveo.nl/networkissues.php, om up to date te blijven met de actuele status.

    ===

    As announced on 30/10/2015 we will preform a Datacenter migration tonight at 22:00 and 08:00 hours CET time. If you are using a dedicated, cloud or colocated server, we kindly ask you to orderly shutdown your server(s) before 22:00 hours. 

    Please visit https://my.serveo.nl/networkissues.php to keep informed by the status.

    ---

    #1 -  22/10/2015
    ---
    Op vrijdag 30 oktober van 22:00 CET migreren wij onze IT infrastructuur naar de Serverius Datacentra DC-I in Dronten en DC-II in Meppel. De migratie wordt uitgevoerd tijdens het onderhoudsvenster tussen 22.00 en 08.00 uur.

    Serveo zal met deze stap de dienstverlening voortzetten zoals u gewend bent en deze uitbreiden met diensten als anti DDoS migitation en IP connectivity. Daarnaast worden hands-ons support tickets versneld verwerkt en breiden we onze netwerk capaciteit uit naar 10Gbit.

    Deze migratie heeft betrekking op alle shared hosting, cloud server, dedicated server en colocated dienstverlening aangeboden vanuit Interxion. De dienstverlening zal tijdens de migratie niet beschikbaar zijn.

    Enkele aandachtspunten:
    - Colocated dienstverlening verhuist naar Serverius DC-I in Dronten. Ons professionele team zal uw hardware met zorg verhuizen en aansluiten. Zorg ervoor dat uw hardware uitgeschakeld is om 22:00 CET;
    - IPv4 adressen blijven ongewijzigd;
    - IPv6 adressen vervallen, u kunt een nieuw adres aanvragen na de migratie via support@serveo.nl.

    Bezoek onze website en NOC pagina (my.serveo.nl/networkissues.php) om de actuele status van de migratie te bekijken. 
    Mocht u naar aanleiding van deze mail nog vragen hebben, dan kunt u contact opnemen via my.serveo.nl of support@serveo.nl.

    ===

    On Friday 30th of October from 22:00 CET we will migrate our IT infrastructure to the Serverius datacenters DC-I in Dronten and DC-II in Meppel. The migration will be done during the maintenance window from 22.00 PM CET till 08.00 AM CET.

    With this step Serveo will continue its services like you are familiar with and we will even expand it with services such as anti DDoS migitation and IP connectivity. Beyond we will speed up the process time of hands-on support tickets and upgrade our network capacity to 10Gbit.

    This migration effects all shared hosting, cloud server, dedicated server and colocation services delivered from Interxion. During your services will be unreachable.

    Important notes:
    - Colocation services will move to Serverius DC-I in Dronten. Our professional team will migrate and connect your hardware with care. Please power-off your hardware at 22:00 CET;
    - IPv4 addresses will remain unchanged;
    - IPv6 addresses will expire, please request your new IPv6 address after the migration at support@serveo.nl.

    Visit our website and NOC page (my.serveo.nl/networkissues.php) to keep informed about the status of the migration.
    If you have any question regarding this mailing, please contact us at my.serveo.nl orsupport@serveo.nl.
    ---

  • Datum - 30-10-2015 22:00 - 31-10-2015 06:40
  • Laatste update - 31-10-2015 05:50
SAN failover timed-out (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Systeem - NX-cluster
  • #2 - 01:10 -  13/10/2015 CET
    ---
    All servers and services have been recovered.
    ---

    #1 - 23:05 -  13/10/2015 CET
    ---
    At 22:55 we have experiened a time-out during the automatic failover procedure of our SAN storage. During this our primary SAN-head failed to rapidly export the ISCSI services which caused a timed-out on the attached LUNs/targets. As the volume was correctly exported from the primary to secundairy SAN-head, there has been no dataloss. Unfortunate as an result of the iSCSI time-out most of VM's have been automatically mounted into read-only mode. To fix this we have proactively rebooted the effected VMs to mount the filesystem correctly.
    ---

    We apologize for any inconvenience this may have caused.

    If there are any issues, please open a support ticket at support@serveo.nl.

  • Datum - 13-10-2015 22:55 - 30-10-2015 17:28
  • Laatste update - 14-10-2015 13:08
Redundancy failure on 185.90.136.0/22 (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - Routing
  • #2 - 11:14 30-09-2015 CET
    --
    The NOC team at Hibernia have manually rerouted the effected subnet. All services have been recovered. Please contact our support department if you have any issues.

    --

    #1 - 10:33 30-09-2015 CET
    --
    Due to yesterdays VRRP failover it seems there is an issue caused in the redundancy on our subnet 185.90.136.0/22. Resulting the IP block being unroutable. We have passed this issue to investigate by Hibernia. 


    --

    Sorry for any inconvenience caused.

  • Datum - 07-03-2036 10:33 - 07-03-2036 11:14
  • Laatste update - 30-09-2015 11:21
Package loss - VRRP AMS5 (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - Core-switch AMS5
  • #5 - 23:05 29/09/15
    --
    During debugging of both VRRP paths we ran into issues on one of both Fiber optic modules. The module has been replaced which restored the traffic flow over the VRRP upinks.

    --

    #4 - 19:20 29/09/15
    --
    Currently the VRRP error rate has increased, hence all services at AMS5 can face package loss. We are investigating the matter.

    --
    #3 - 14:20 29/09/15
    --
    Our network enineers have update the coreswitch configuration, which should improve the VRRP configuration back to normal operations.

    --

    #2 - 22:04 28/09/15
    --
    The error rate on the VRRP uplinks are restored back to a tolerant level. Our network engineers are investigating the issues.

    --

    #1 - 20:45 28/09/15
    --
    Due to ongoing errors on our core uplinks we are facing package loss. All services at AMS5 are effected 

    --

    Sorry for any inconvenience caused.

  • Datum - 28-09-2015 20:45 - 29-09-2015 23:05
  • Laatste update - 30-09-2015 02:11
Automatic SAN-failover failed (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Server - Cloud-01
  • #3 - 00:09 28/09/2015 CET
    ---
    Due to ongoing perfomance issues we have rolled back to our primary SAN-head. All services and loads have been restored on normal level.

    #2 - 20:00
    ---
    As a result the amount of FSCK tasks we are facing a higher load troughtout the cloud platform. Due to this it is possible you are having time-out on request or connection drops. We forecast that this bahavior will be stabalizing this evening.

    ---

    #1 - 18:55
    ---
    At 18:05 we have experiened an cache fault which triggered the automatic failover procedure on our SAN. During this our secondaire SAN-head failed automatically ran the import command. As the volume was correctly exported on the primary SAN-head there has been no dataloss. Unfortunate as an result of the connection iSCSI loss had been greater then the time-out time most of VM's have been automatically mounted into read-only mode. To fix this we have proactively rebooted the effected VMs to mount the filesystem correctly. Around 18:38 CET we have finished the reboot initiation for all VMs.

    ---

    We apologize for any inconvenience this may have caused.

    If there are any issues, please open a support ticket at support@serveo.nl.

  • Datum - 07-03-2033 18:05 - 28-09-2015 00:09
  • Laatste update - 28-09-2015 00:10
High platform loads (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Server - Cloud-01
  • #3 - 02:03
    All services & servers have been recovered on normal loads. Our engineers will continue to monitor the systems.

    If you have any problems, please open a ticket with us at support@serveo.nl


    ---

    #2 - 01:15
    While we are working on a solution the loads and connectivaty should be stabalizing at this moment. 

    ---

    #1 - 00:45
    Currently we are facing a high load troughout the our cloud server platform. Our engineers are working to identify the cause and solve it.
    All cloud services and shared hosting services are effected by this issue.

    ---

    Sorry for any inconvenience caused.

  • Datum - 15-09-2015 00:43 - 15-09-2015 02:15
  • Laatste update - 15-09-2015 02:06
Incoming DDOS (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Overige - Network wide
  • In the previous 45 minutes we received an incoming DDOS of 10Gbit on our main location at AMS5.

    This caused an high latency on all services and custeromers may have experienced packages loss during. Destination source has been nul routed. All services have been recovered.

    Sorry for any inconvenience caused.

  • Datum - 08-05-2015 21:30 - 08-05-2015 22:15
  • Laatste update - 08-05-2015 22:33
Technical web03 issue (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Server - web03
  • #3 - 20:03
    ---
    Problem has been marked as solved. For any support questions please contact support@serveo.nl or open a ticket using my.serveo.nl.

    #2 - 19:53
    ---
    Our sysadmin has identified the problem and is expecting an recovery soon.

    #1 - 15:40
    ---
    Currently we are seeing a technical problem on web03 which is causing a downtime for all hosted websites on this server. We are working to fix this issue.

    -
    Sorry for any inconvenience caused.

  • Datum - 30-01-2015 15:40 - 30-01-2015 20:03
  • Laatste update - 30-01-2015 20:03
Upgrade on cloud cluster (Opgelost)
  • Prioriteit - Gemiddeld
  • Storingen Overige - Cloud server management
  • #2 - 29/01/2015 - 14:21 CET
    ---
    Upgrade has been sussesfully completed.

    #1 - 29/01/2015 - 10:00 CET
    ---
    Currently we are preforming a update on our cloud cluster managemnet network. During the upgtade managment functions using My.Serveo.nl are not available, the VM's are not affected and no downtime is forecasted.

    -
    Sorry for any inconvenience caused.

  • Datum - 29-01-2015 10:00 - 29-01-2015 13:00
  • Laatste update - 29-01-2015 19:04
Web02 (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Server - web02
  • Op dit moment ondervinden we een storing aan web02. We zijn de server nu aan het nalopen en geven z.s.m. een update.

    #Update 1 - 10:40
    Server was out of memory en is gereboot. Alle services zijn weer up.

    Onze excuses voor het ongemak.

  • Datum - 03-12-2014 07:20 - 03-12-2014 10:40
  • Laatste update - 03-12-2014 10:42
Storing web01 (Opgelost)
  • Prioriteit - Kritisch
  • Storingen Server - web01
  • 22/04/2014 10:05:47
    De storing is nagelopen, er bleek een memory issue te zijn. Alle services zijn inmiddels weer normaal beschikbaar.

    21/04/2014 01:00
    Op dit moment ondervinden we een storing op web01. Hierdoor lijken alle services niet te responden zoals gebruikenlijk. We zijn on-site om dit probleem op te lossen.

    Excuses voor het ongemak.

  • Datum - 21-04-2014 01:00 - 22-04-2014 10:05
  • Laatste update - 22-04-2014 23:33

Server status

Hieronder vindt u een real-time overzicht van onze servers, waar eventuele bekende problemen vermeld staan

Server HTTP FTP POP3 PHP info Server belasting Uptime
web03 PHP info
web05 PHP info
web06 PHP info
web10 PHP info
web11 PHP info
web12 PHP info