72 hours later and a huge amount of progress!

I have worked about 60 of the last 72 hours and while normally (due to being a little bit autistic) I enjoy typing numbers into little boxes (hey! Don’t judge lol!), at this point even I’m finding the fun element is starting to wane a little bit. :D

This is roughly were we are at after 72 hours.

Critical

Until all the critical jobs are done it's either non-functional, or, bits might not work correctly. Stuff like customer details, order headers, etc. - That's 80%-90% done.
I can’t mess around changing data by doing things like allocating stock, etc until it’s 95%+ done. The odd thing I can manually correct as I go, but, it’s quicker to move the data first before I start altering it.

Urgent

Moving all the product listings etc so people can buy things, but, also that's how I track stock.
I've moved across about 1,200 of the 3,300+ types of corals, inverts, fish etc, which is about 1/3 the total number, and those ones account for 50% of what people buy.
This is quite important because until all the products are on the new system there is no way to track stock etc. This is the biggest part of the operation and we’re close to the halfway point.

Non-critical stuff

Improving tagging, tidying up broken links etc. That can be done in bits and pieces over the next few months, that only really affects Google ranking and doesn't cause any major issues really.
I can do this later, it doesn’t affect day to day operations much.

When will things all be functional:

Critical stuff

Booking out deliveries, packing boxes, order amendments, wholesale runs etc.

If I only had to do this then maybe 24 hours (ish), but, I am going to need some sleep, and there are some other jobs (admin, fragging, work on the systems, shopping, etc) that I’ve mostly been able to stall over the weekend, but, are going to need doing soon. So, let’s say 48 hours ish.

Taking new orders - This is now operational. - Online now.
Discount codes - Currently not operational, should be working later today - Monday evening/night.
Allocating stock and booking out deliveries for this week - Starting from Wednesday night.

Urgent stuff

Getting all the stock codes moved over for stock control and so people can buy things.

There are a lot of stock codes, but, quite a lot of those are for things people rarely buy, seasonal offers, etc.
Currently around 50% of what people spend money on is online on the new site. - 50% online now.
By Friday almost all the important stuff should have been moved - 75%+ by Tues/Weds, 90%+ by Thurs/Fri.

Other stuff

This mostly only affects Google ranking and doesn’t interfere much with 99%+ of things. I’ll get round to this later.

New!

Make sure you take a look at the new site, it’s a huge improvement. You can make purchases etc on there, that all works fine because everything you can see on the new site has already been moved. The only things that are affected by the move are things that I haven’t moved yet, so stock that is only on the old website and not on the new website yet etc.

Click HERE to go to the new site

Keeping it simple.

All new orders will be on the new website. They have a different numbering system, codes etc.

All orders up until 3pm last Friday will stay on the old website (it stops them getting confused by any stock codes that change etc), it also makes it far easier to identify these oldest orders and get them tidied up as soon as possible.

Old orders will remain on the current mostly paper based system. New orders are almost all electronic, so keeping them as two separate things saves having to type all the old orders into the new system.

I wanted to switch for two years, but, it's a big job. Some of it is automated now so it's a bit easier than it used to be but the amount of data to move is vast, and quite a lot of things need manually correcting from one layout to another etc.

The reason for creating a new site at the same time, is that by moving the checkout onto Shopify I have to move all the customer accounts and stock onto Shopify, and at that point it’s quicker to build a new site with clean blank pages, than mess about making the two work together.

In total to move are around:

4,600 customer profiles,
11,000 orders,
7,000+ product SKU's each with 20 or so data entries each
20,000 images, which each require titles, tagging etc.
Several million words of text, at best guess around 8 million to 10 million words (for comparison if you total all the words in all the Harry Potter books that's just over 1 million).
Over 4,000 web pages, each requiring URL redirects.
700 Blog posts (with yet more words and images).
Plus external links, Google listings etc, automated email systems, new checkout system with new card handling accounts etc, different accounting system, several hundred discount codes, meta tagging new pages, etc.

Next update will be in 24-48 hours ish.

James

Half the website is now operational already, including all WYSIWYG corals :)

Update: 5:20pm Sunday 01/10/23.

All products marked with a green dot like this “🟢” can now be purchased using our new Shopify checkout system (look for the new cart button). Any products that don’t have a green dot cannot be purchased yet.

We should be on track to have pretty much all the products available again within the next 24-36 hours.

James

All WSYSIWYG’s now online!

Update: 11:00am Sunday 01/10/23.

I’ve just now added all the rest of the WYSIWYG’s in stock.

Click HERE to see all the WYSIWYG's.

🟢This green dot

Update: 1:00am Sunday 01/10/23.

All products marked with a green dot like this “🟢” can now be purchased using our new Shopify checkout system (look for the new cart button). Any products that don’t have a green dot cannot be purchased yet.

Oops we crashed the server!

Update: 6:00pm Saturday 30/9/23.

A very quick update on where we are at. Because some of this is quite technical I'm splitting it into two sections "Current status" and "More information".

Current status

Ok so currently all the website is still online, apart from the payment screen which is still offline (see 1 below for more info).

What happened?

This is only about the 4th time in 3 years that the payment system has crashed out due to too much traffic at once (2).

The reason it crashed was because lots of people checking out at precisely the same time (2). Normally even on really busy days such as Boxing Day it can cope, but, because of the way the Pay Day Offers worked, plus the time of day and day of the month, that's why it crashed then, when normally it copes on days that overall are many times busier (2 & 3).

On Monday I could get Google to allocate us a bigger server space for checkout (we've had to do this twice before). (4)

Some good news!

However, there is a better solution which is for me to change the checkout section over to another provider. I've wanted to do this for ages anyway.

It's a big job, and you have to take the ordering system offline during the switchover, so for many hours, possibly several days whilst it's being switched over we can't add new stock, adjust orders, take new orders etc, which is a pain and why I've put off doing it as there is never a good time to put everything on hold to begin it and I can't do almost any other jobs until it is complete and online again.

Example:

Imagine say you're having a major building work job on your house which would mean for 72 hours you couldn't leave your house, but, at the same time you had no beds, no toilets, and no roof. No matter how much better your house would be afterwards, there's never really a good time to be trapped in it with no beds, toilets or roof!

However with the payment screen down this is actually, by chance, the perfect time to make the switch (4 & 5).

While I'm switching everything over I can't do much else, because I can't alter stock numbers etc until it's moved over. I can do this in chunks, so for example I'll probably move the WYSIWYG corals over first. That might be finished tonight, so later tonight (all being well) you should be able to add WYSIWYG's to cart and checkout etc using the new checkout system, but, you won't be able to add anything else to that cart until that has also been moved. Then bit by bit over the next 12-72 hours (ish) everything will pop back up one by one.

Some things will be different, I haven't done this before so I'm not certain which bits will change, but, things like the Buy button might be a slightly different shape, the boxes you put your address in etc will look slightly different, new orders may have a different numbering system, your account screen might look different, it might ask you to create a new password (maybe), but, most things will look the same.

Changing it over is actually probably the quickest way to get it online again, and I've wanted to do it for ages because it brings lots of benefits, and with everything currently offline anyway it's the perfect opportunity to do so.

New things!

To start with the new system will work pretty much the same, but, once it's all moved over I can bring in loads of new features that are available on Shopify that weren't available on Squarespace. For example I can bring in things like search filters.

With search filters you can then click "Search" and then select options such as "show me all Zoas in stock that are blue and cost under £20" and then it will search the whole website and pop up on screen just those ones. So you will be able to quickly find corals of a particular type, colour, price whatever, you just select what you want to see and it puts them all on one page for you, rather than you having to wander around the website finding it.

Shopify also lets me filter orders so I can say ask it show me all orders that are waiting for 100ml Coral Food 99 and it will pop them all up on screen for me, rather than the current system of me having to read every product line on every paper copy of every live order and jotting them down on a piece of paper. As you can imagine this saves me a massive amount of time on admin each week and when I am ordering stock I know exactly how many we need of each item and who they are for, rather than either spending 10 hours going through hundreds (sometimes thousands) of sheets of paper and keeping a tally, or, just having a best guess. As you can imagine this will save vast amounts of time, be much more accurate, and vastly reduce waiting times on many items!

When will it start working again?

I should (hopefully) have the first chunk of the move done later tonight,. and will update this when it's done.

Sorry for the disruption everyone. Some things should start working again later tonight at some point, and we should get to 98%+ working by Monday some time at best guess, and then I can get back to booking deliveries out, and other normal stuff.

Oooo!

Later next week (maybe sooner), some new improved features will start popping up as I figure out what I need to set to switch them on. This will make it far easier for you to find things, and everything should load far faster too. It will also vastly reduce the amount of time it takes me to add products, do ordering, etc. Which will mean a larger choice of things for you to buy, about a 50% increase (maybe more) in the amount of orders we can send out each week, and much shorter wait times on out of stock items too. Yay!

Having the checkout system crash and go offline yesterday seemed like a massive problem, whereas actually it might turn out to be the best thing that's happened to me this year!

James

More information.

1- Payments are on a separate server which is why that bit doesn't work but the rest of the site does. Most websites operate that way.

2- The reason it crashed out was the Pay Day Specials were really popular and so many people tried checking at once and crashed out the server. Now this isn't because of a lot of orders in say 1 day or even 1 hour, it can handle up to several hundred orders per hour, because taking the payment for each order is only a few seconds of server time. Where the problem happens is when say 10 people all try at the same 10 seconds. If 10 or even 50 people are spread out over 1 minute it can handle that, but, not many all at the same few seconds.

3- On say Boxing Day the orders were spread out a bit more over the day. Plus multiple companies have their checkouts on the same server, so while we were really busy on Boxing Day, many companies might have been quiet, or closed for Christmas, so on Boxing day the server could lend us some of their space allocation during our busiest moments. However, this time it was a Friday (one of the busiest days for web use), it was also the weekend most people get paid (so more people shopping online than a normal Friday), and it was 3pm (one of the busiest times on web being school finish time, companies sending out payslips, weekend promotional material, etc, basically everyone doing stuff at exactly the same time). Some people also get paid at 3pm, so that is the exact time they may complete an online purchase they had already added to cart etc.

4- Currently this is how the website is split:

Main pages, Navigation, Blog pages - Squarespace which is part of Google - all on Server A.
Product details, Stock amounts, etc - As above, on Squarespace on Server A.
Customer account screens - As above, on Squarespace on server A, but, the secure details (phone numbers etc) are on Squarespace on a higher security Server B. Server B I have no direct access to, that way I cannot see sensitive data such as your passwords, full card numbers, etc. This is standard practice for all card transactions worldwide.
Payment screen - On Square space server B, payments handled by PayPal (PayPal handle card payments not just PayPal payments).

During this part of the switchover all the main pages etc will stay on Squarespace, but, the products and everything on Server B will be moved over to being hosted by Shopify. A lot of websites work this way.

Example:

Think of it like this. In say a big Tesco store, Tesco own the land, the building, all the shelves, stock, etc. However they don't build the tills, they buy tills and put them at their checkouts. It's a bit like that. #

So the switchover is a bit like Tesco taking out the make of till they had and putting in new tills made by someone else. So we had PayPal tills which were maintained by Squarespace, whereas now we will have Shopify tills, maintained by Shopify.

So if Tesco did that all of Tesco would still be the same, but, the tills might have some extra buttons, and the receipts they print out may have a different font for example. Our website switch is essentially the same. All the website will look basically the same, but, at checkout some buttons might be different and your email receipt will look slightly different, everything else is mostly the same.

5- With the payment screen offline (so no orders can be taken), no promo work to do (there is no point doing promotional work if no one can buy anything), no stock coming in (it's the weekend so the wholesalers are closed), and no deliveries going out (I can't do deliveries today and tomorrow because the website has to be fixed first)

Previous updates below.

Update: 6:49pm Friday 29/9/23.

Hi everyone,

Due to the large amount of people all trying to checkout at the same time it's overwhelmed the server and taken the checkout screen offline, the Pay Day offers were a bit too popular lol!

Pay day specials! - The offers that were so popular they broke the website!

Some of you may remember we had this happen once before when we launched our Big Easter Sale in 2020, when so many people all tried checking out at once and took the checkout screen down then too. Our payment screen is hosted by Google, and they upgraded us to a bigger server allocation so we could handle more people checking out at the same time. Since then it's just about coped, but, with it being pay weekend this weekend, and a really good Pay Day Specials offer it tipped it over the edge and crashed out Google's server (oops).

Last time it took a couple of days to get it back online, because each time I switched it back on everyone rushed to checkout straight away and so each time it just went straight down again within minutes (Google were not best pleased with that). In the end I had to wait for them to allocate more server space and then bring it back up at 4am so it didn't get swamped as soon as it was online again.

So you can still use all the website, and add to cart etc, but, you won't be able to get to the payment screen. So feel free to pick out what you want, add it all to cart etc. All the other 7,700+ pages work except the checkout basically.

Once it's up and running again I'll notify everyone in blocks so everyone doesn't all checkout at once, and I'll extend the Pay Day Special offers so that no one misses out. As it's now Friday night, it's likely to be offline until sometime Monday I reckon. I'll update you when I find it how long it'll take them to allocate us more server space. I'm going to be flat out trying to fix this all weekend I reckon, so all other jobs (except looking after animals) will have to be put on hold until I've fixed it.

Sorry for the delays, I'll get it back up and running as fast as I can.

James