Andy Crouch - Code, Technology & Obfuscation ...

Migrating Stand Alone G Suite Account Into Multi-Domain G Suite Instance.

Photo: Unsplash - Benjamin Dada

This week I migrated a G Suite account from being a stand-alone instance into a multi-domain account. Given that the account was just email, calendar and drive I was expecting it to be easier than it was. The documentation was OK if you find what you are looking for but I didn’t find a single cohesive guide. So, here it is.

Background

I had a G Suite account that was tied to one domain, let’s call it source.com. I had a second G Suite account that had been set up to manage multiple domains (a much-ignored feature from what I can see online). We’ll call this domain target.com. In order to reduce costs, I wanted to migrate and manage the source.com users and associated data from within target.com .

First, the good news. Google makes it easy to transfer the data between two G Suite accounts. Now, the not so good news. You will need two spare domains to complete this process. Oh, and Google does not provide a way to transfer Drive files from what I can see. Finally, you will need time to prepare and a lot of patience.

Preparation

The preparation is the most time-consuming part of this process. Having verified the steps with a Google Support guru I am assured this is the most fail-safe way.

So here goes:

  • First you will need to disassociate the users in source.com from the domain. If you have a secondary domain tied to your source.com account then you can move to the next step. If not then you will need to navigate to the Domain section of your account. There you will need to add the spare domain you can use to free up source.com. I used source.co.uk. To complete this, click the Domains menu item in the top level G Suite menu and follow the onscreen instructions.
  • The next step is to Rename all your users so that instead of them being a.user@source.com they will become a.user@source.co.uk (or whatever spare domain you have used). You will do this by going into each user and clicking the Rename User option on the left, just under the user’s details.
  • The last task as you are going through your user list and renaming them is to reset their passwords. You will need to do this as during the migration you will need to know each users password. You can do this in the same user screen as above using the Reset Password option.

Now you have updated all the users and the domain in source.com you will need to add the second spare domain to target.com. I used target.co.uk and the process is the same as above. Login to target.com and click the Domains menu item on the opt level menu. Follow the onscreen instructions.

One final task to complete is to make sure that source.com is no longer the primary domain in the original account. Log into source.com and navigate to the Domain section and make the spare domain that you have set up the primary domain.

At this stage, you should have your users in source.com updated to source.co.uk and have reset their passwords. You will have also set up the spare domain on target.com.

Migrate Data

To set up the migration service that G Suite provides click the Data Migration menu option in the G Suite home menu.

  • On the next screen select “Set Up Data Migration”.
  • Under Migration Source, select G Suite and click Start.
  • In the next screen set the configuration settings as you want.

The next screen is where you set up the user mapping and kick off the user’s migration.

Click Add User and you will see the following screen:

In Source Email field add the account name in source.com which should now be username@source.co.uk (or whatever domain you used). In the G Suite Email field add the user name and use the spare domain you set up in target.com. And finally, set the password and use one that you know as you will need it to migrate the Drive data later. You can click start and the migration process will kick off for that user. Now you just need to repeat n times the number of users you have.

One you are at this point you just have to wait. Coffee time!

Transferring The Domain

Once the Data migration service is complete you can now transfer source.com to target.com for management. The process is easy:

  • Log into the original account and navigate to the Domain section and click remove next to the source.com domain.
  • Log into target.com and navigate to the Domain section and add and verify the source.com domain.

Applying The Original Domain In You Multi-Domain Account

Now that the source.com domain has transferred over to target.com you can update the users added as part of the migration to use the source.com domain. The process is the same used above in the first section:

  • We need to rename the users from the spare domain we used to migrate back to source.com. You will do this by going into each user and clicking the Rename User option on the left, just under the user’s details. You will then select the source.com domain and save the changes.

At this point all G Suite email, Calendar and Contact data should be migrated.

** DO NOT DELETE THE ORIGINAL ACCOUNT YET **

Transferring G Suite Drive Files

The Data Migration service offered by G Suite does not handle Drive files, as I mentioned above. The easiest way I could find to transfer files for minimal cost was to use Insync which is a Google Drive client that supports Windows, Linux and Mac OS. I have used Insync for years as a Linux client and it works really well.

To set up the Drive migration you will need to log into both the original source.com and the new target.com accounts as the user, trigger a sync to a local machine and then manually copy the files from the source.com directory on the local machine to the target.com directory and let it sync up.

Finalising The Migration

At this stage all data should be migrated between the two accounts for each user and the final step is to reset all of the new user account passwords under target.com. You can then allow you users to log in and verify the migrations success.

Once all users have verified that there are no issues you can then delete the original source.com G Suite account.

Wrapping Up

This is a pretty manual process which works fine for a small number of users. I migrated fifteen. There are programs out there that will automate the process for you at a cost. For my client the costs were prohibitive but the benefit was massive.

Managing multiple Domains via G Suite doesn’t seem to be that well written about on the web. For agencies and consultants I suspect it is an ideal way to segregate your accounts for clients and provides a useful way to set up permissions using Organisational Units and Groups. I will try and write more about these subjects soon.

If you have any questions around the process then let me know via twitter or email.

Moving To Remote Working Due To COVID-19

Photo: Unsplash - pixpoetry

COVID-19 has swept through countries and had a direct impact on every person. Whether it be through contracting the virus, economic repercussions or social distancing, our way of life has been significantly changed. As I write this we still do not know what is required to get life back to a form of normality or how long that will take.

One area that, through the effects of this virus, has seen immense growth is remote working. The number of companies offering remote roles has been growing for years but with the spread of COVID-19 it became an overnight necessity for all non-essential office-based workers. This has generated a huge opportunity for these companies to try a new way of working. The time frame that these companies had to get their operations working remotely may mask many of these benefits.

The aim of this post is not to rehash the many (too many) posts on how best to work from home. Its aim is more to highlight both technical and cultural issues you need to consider.

Let’s start with the technical, moving all your employees out of the office is no easy task. Most companies these days provide laptops for convenience. This means your staff will generally have a machine that you can control to work from home on. This means that you can control the basic’s of security like anti-virus protection (irony huh?) and VPN connectivity. Where ever possible you do not want your team working from the family laptop. If they do you need to consider the impact on your data and the vulnerabilities that may expose. It would not be unfair to insist that any non-provided computer that an employee works from is upgraded with your corporate security solutions. Next, you need to consider their ability to communicate. So far the mobile networks (at least in the UK) have stood up to the increased load. So it would seem, have the broadband providers. The thing to consider is that your employees may not have high capacity lines. Speeds and transfer rates vary and if you employee has a family that is also consuming Netflix and PlayStation games due to being off school, that can have an impact on their ability to video call and work. Not every situation is the same and what is a small cost to absorb to upgrade a couple of the teams broadband for a small company is a much larger cost as your headcount increases. It’s something to consider.

The next main thing to consider is your data and how secure it is with the sudden move to remote working. Thanks to GDPR you should have a full map of all the data points in your organisation and which systems those points relate to. Many systems are cloud-based these days and the preparation for GDPR would have meant ensuring your providers meet the regulations. If you are not using a service like GSuite or Dropbox to manage your files then how are your employees accessing the data? Your main concern here is that data and documents do not get shared via insecure services or personal accounts. If you are running your file-sharing infrastructure then the least you need to ensure you have in place is VPN connections for all employees to your file servers. Remember that no regulation or data breach will be forgiven in 2020 just because of COVID-19. Check your data audit and secure all access.

The next obvious thing to consider is communication. There are a host of options but ensuring that your employees use the right ones is key. Slack, Microsoft Teams, Facetime, these are all proven and reliable forms of communication tools. For video conferencing, there are endless options but selecting one can be trickier than it should be. Zoom is a massively popular video calling app but has a history of security issues. Again, take time to review solutions and their history before getting the whole team on board. Slack for me has pretty much all you need to stay connected. How well you use it is another article but it has calling and video chat, screen sharing and most apps provide a way to integrate with it now.

Communication brings me to the final area I want to cover in this post, team culture. Every company has its culture and without realising it that can be governed in part by the locations of your teams. Office-based teamwork tends to be much more synchronous than when your team is remote. This is obvious as you can walk up to any of your team and have a conversation in real-time. The employee can understand context and meaning from you. These are elements that can be lost in a text-based conversation.

You can see when that employee is at their desk so you know when to have that conversation. When a team becomes remote you need to adjust this element of the culture. There needs to be an element of flexibility. This is not so your team can kick back and catch up on Netflix and fit work in when they want. When someone transitions to working from home it can add an element of anxiety for some staff. They feel they have to respond to a Slack message immediately. They don’t feel comfortable taking an hour for lunch as they fear how that might be perceived. The level of anxiety across your team may differ but it will be there in some form especially now at a time when they have been forced into remote working. Add this sudden change, anxiety and wider concerns about the world events you will find that it will take some time for your team to settle into being remote. You or your management team must make time to check in with your employees and ensure their OK. Agree as a team to the standard you now expect. It’s fine if you need your core team available between 8:30 and 5. Share that you want an all-hands team call on a Wednesday at 10am. Figure out what you need as a business but be sure to adapt to a more asynchronous way of working and be slightly more forgiving of your team during this unprecedented event.

One last tip on culture. Add some fun elements to it. Arrange a virtual team lunch, add a music sharing channel to Slack, arrange the book club virtually. This is key to helping the team adjust to the sudden change. Be visible as well. If you are the CEO, sales manager, a developer, whoever just be visible and friendly and accessible.

I hope this article has provided some points to consider. I didn’t want to write another “don’t work in your PJ’s” article as people should know that already. If you have any questions around going remote, building teams or culture then let me know via twitter or email.

Thoughts on "What You do Is Who You Are"

Photo: Ben Horowitz

Earlier in the year, I wrote some thoughts on company and team cultures. The brain dump can be found here. At the end of that post, I explained the subject had been floating around the grey matter as I had picked up a copy of “What You Do Is Who You Are” by Ben Horowitz. Having completed the book I wanted to share my thoughts about the book.

The book is not a standard walk through on how to implement culture within your company. Instead, it takes a long trek through three specific historical characters and samurai history. While the samurai don’t stand out, the three historical characters do. Toussaint Louverture, Shaka Senghor and Genghis Khan. None of these figures I knew much, if anything in Senghor’s case, about. But, having followed Ben’s writing for some time I did know the man knows a lot about three things: Rap music, Business and History.

After asking the obvious question “What is culture” Ben spends a couple of chapters on Toussaint Louverture. Louverture was the leader of the Haitian slave uprising in the 18th century who understood that the culture of slavery was shaping the salves behaviour. He could see an opportunity to change that culture to enable a successful uprising and Ben outlines how he did it. Tactics used include “Keep what works”, “Create shocking rules” and “Walk the talk”. Some of this is obvious, although “Keep what works” is the result of constant cultural development. “Create shocking rules” might not relate as you would think. Here Ben summarises them as memorable, easy to explain and encountered daily. Amazon’s meeting process, Facebook’s early “Move Fast and Break Things” mantra and partnerships that are always 49/51 against the company are used as examples. He provides context against each point backing them all with a solid reason.

The book then flows into the success of the samurai’s and the code they lived by, the samurai bushido. This section really focuses on the difference between values and virtues. These are well documented and Ben shares examples of virtues he’s instilled at companies. One example is that the company “… tell the truth even if it hurts … we do not withhold material information or tell half truths. Even if the truth will be difficult to hear or to say, we err on the side of truth in the face of difficult consequences”. This is obviously, taken from Andreessen Horowitz but he also provides examples from the Netscape days such as “If you see a snake, don’t call committees, don’t call your buddies, don’t form a team, don’t get a meeting together, just kill the snake”.

The book then moves on to Shaka Sengor who in 1991 was convicted of murder. During his time in prison, he became the leader of The Melanics who he transformed so that “when we went back to the community, we could help fix it for other kids.” He clearly transformed himself by his desire to move away from where he grew up and discovered on his journey that culture is not a “set it and forget it” task. A leader has to subscribe to the cultural elements he prescribes or else there is not culture.

The last character that Ben use’s to highlight strong culture is Genghis Khan, not usually high up the business idol worship list. Khan has gone down in history as perhaps the greatest warrior but Ben focuses on inclusion within his army and how he communicated new practices. The element around inclusiveness and diversity is really interesting and Ben is clear to state that getting it right doesn’t fall to a sole “Head of Diversity”. He argues that teams should hire from different background and different talent pools. He uses Khan’s ability to assimilate his enemies armies upon defeat to his advantage as an example.

The book winds up stating that the main virtues you need in modern business are Trust, Openness to bad news and Loyalty. These are not the only ones as you should decide what is important to you and your team but they are core ones that if not adopted can have dire consequences. He is clear that defining questions which have answers that define who you are will help you to define your culture. “Can this phone call wait til tomorrow?” “Should I go home at 5pm or 8pm?”, “Is winning more important than ethics?”.

The book was a really enjoyable read. I love the mix of history and examples and found a few “aha” moments. Does it help you go and set a culture for a team or an entire company? No. Instead, it makes you think about culture at a much deeper level. You may take nothing actionable away from this book but it makes you think hard about how to get your culture right.

If you have any comments or questions around building teams or culture then let me know via twitter or email.

JavaScript Destructuring Syntax

Photo: Unsplash - Markus Spiske

JavaScript has the destructuring assignment syntax to allow you to unpack objects and arrays into variables. I still see a lot of code where either the developer is not aware of this language feature or just didn’t use it. The feature was added in ES6 and enables more terse code without the lose of intent. What follows is a whistle-stop tour of the syntax and how to use it.

Array Destructuring

Array destructuring allows you to define variables based on the position of elements in an array. A simple example is:

const numbers = [1,2,3,4];
const [ one, two, three, four ] = numbers;
const [ a,,e,g ] = numbers;

console.log(one);
console.log(two);
console.log(three);
console.log(four);

console.log(a);
console.log(e);
console.log(g);

To destructure array values you will see that the variables are defined by declaring them in a set of square brackets. They are defined in the order they will be mapped to against the array elements. If the variables do not already exist then you need to prefix the declaration with a let or const. As the position is used to destructure the values from the array you need to handle the position of values you do not want. You will see there is an empty space declared in the a,e and g destructuring.

If you are interested in the first couple of elements in the array then you can apply a “rest” pattern and combine the remaining elements into a single variable. This is achieved by prefixing the final variable declared with three periods as shown in the example below:

const numbers = [1,2,3,4];
const [ one, two, ...remainder ] = numbers;

console.log(one);
console.log(two);
console.log(remainder);

You can us this array destructuring approach with iterables as shown below:

function* makeRangeIterator(start = 0, end = 100, step = 1) {
    let iterationCount = 0;
    for (let i = start; i < end; i += step) {
        iterationCount++;
        yield i;
    }

    return iterationCount;
}

var [first, second, third, fourth, fifth, sixth, ...rest] = makeRangeIterator();
console.log(sixth);
console.log(rest);

Object Destructuring

Destructuring objects works in a near-identical manner to what we have seen above. Instead of a variable being bound on position, they are bound to object properties as shown below:

const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
    address:{
         houseNumber: "14446",
         street: "Looney Road",
         town: "Loonery Town"
    }
}

const { firstName: first_name, lastName: last_name } = person;

console.log(first_name);
console.log(last_name);

An even clearer approach is to use the provided shortcut which works if you name the variables the same of the properties such as:

const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
    address:{
         houseNumber: "14446",
         street: "Looney Road",
         town: "Loonery Town"
    }
}

const { firstName, lastName } = person;

console.log(firstName);
console.log(lastName);

You will notice that instead of square brackets, object destruction used curly brackets to surround the variable declaration.

In order to deconstruct more complex objects such as the address data in the person object you can nest the declaration of variables in line with the structure of the object. You prefix each block of variable destructuring with the name of the parent property and surround their child declarations with further curly brackets as shown below:

const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
    address:{
        houseNumber: "14446",
        street: "Looney Road",
        town: "Loonery Town"
    }
}

const { address: {houseNumber, street} } = person;

console.log(houseNumber);
console.log(street);

You can nest your destucturing of the data as deeply as you want by continuing the pattern as shown here:

    const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
    address:{
        houseNumber: "14446",
        street: "Looney Road",
        town: "Loonery Town",
        phones:{
            mobile: "03456789",
            landLine: "23456789"
        }
    }
}

const { address: { houseNumber, street, phones:{ landLine } } } = person;

console.log(houseNumber);
console.log(street);
console.log(landLine);

Destructuring Defaults

If you try to destructure array elements or object properties that don’t exist your variable will be set to undefined as shown below:

const person = {
   firstName: "Donald",
   lastName: "Duck",
   age: "105",
}

const { weight } = person; // <- undefined


console.log(weight);

If you are unsure of the properties you are destructuring you can set default values when declaring the variables such as:

const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
}

const { firstName, lastName, height = "110" } = person;

console.log(firstName);
console.log(lastName);
console.log(height);

Wrapping Up

Hopefully you can see the benefit of destructuring and how it reduces the number of variable declarations you make to retrieve data from objects and arrays. You can use the syntax in a few ways other than just variable declaration.

You can define a function to accept a single object as a parameter and use the destructuring syntax to pull the actually values from the object that you need such as:

const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
}

function printNameFor({ firstName, lastName }){
    console.log(firstName);
    console.log(lastName);
}

printNameFor(person);

You can also use the syntax to handle returning multiple values from a function such as:

const person = {
    firstName: "Donald",
    lastName: "Duck",
    age: "105",
}

function getNamesFrom(person) {
    const { firstName, lastName } = person;
    return [ firstName, lastName];
}

const [ firstName, lastName ] = getNamesFrom(person);

console.log(firstName);
console.log(lastName);

If you have any questions around the destructuring syntax then let me know via twitter or email.

GraphQL, PostgreSQL & Hasura (Pt1)

Photo: Unsplash - Isaac Smith

In a recent project, we decided to build our backend in Hasura. This was my first time working with it and I have been impressed with the ease and power it provides. Essentially, Hasura melds the GraphQL language with PostgreSQL database to provide easy and fast real-time API’s powered by your data schema. In this post, I will cover GraphQL (at a high level) and how Hasura makes it easy to set up an API in no time.

(This is based purely on my recent experience and I am not in any way affiliated with Hasura).

GraphQL

GraphQL is an open-source query language that was developed by Facebook. It was released publicly in 2015 and is designed to power API’s by providing a runtime that allows clients to query just the data they need without the complexity and baggage of something like an ORM. Its flexibility means it makes evolving API’s overtime easier and it speeds development by removing the need to generate as much boilerplate code. For example, using an ORM like Objection.js you might write something like the following:

const person = await Person.query().findById(1);

console.log(person.firstName);
console.log(person.lastName);

This means that you are returning the whole Person record to be able to generate an object which the data is mapped to. You are then just using a small subset of the fields on that object. With GraphQL you can query just the data you need rather than having to return a whole record. The above query in GraphQL could be written as:

query{
  person{
    firstName
    lastName
  }
}

Which returns

{
  "person":{
    "firstName": "Donald",
    "lastName": "Duck"
  }
}

This is a trivial example but shows the power behind GraphQL Ask for what you want and get the response as JSON. The GraphQL language includes everything you need to create, read, update and delete data. Query’s, such as the snippet above, allow you to read existing data, Mutations allow you to create, update and delete data and Subscriptions allow you to monitor part of your schema to receive real-time updates.

Rather than provide a walk-through of GraphQL top to bottom I recommend you read the excellent tutorial on GraphQL.org.

PostgreSQL

PostgreSQL doesn’t really need an introduction as it really is the most advance open-source database available. If you have worked with SQL Server or a MySQL derivative then you will be at home. In the way in which Hasura works you do not need to interact with PostgreSQL directly. If you do want to learn more about it then the documentation can be found here.

Hasura

Setting up a playground to test Hasura is very easy. You can deploy an image to Heroku on their free tier and be up and running in minutes. Once deployed you will have a PostgreSQL database and an endpoint with which to access Hasura. I followed the tutorial for building a todo app which can be found here. I should mention I have found their documentation really good.

Once you are up and running you access the UI from your browser.

There are 4 main tabs to the UI labelled:

  • Graphiql
  • Data
  • Remote Schema’s
  • Events

All running GraphQL instances provide graphiql which is a repl type environment in which you can build and test your queries, mutations and subscriptions. The Hasura version is standard and provides automatically generate documentation and point and click query building capabilities.

The Data tab is where you will design and build your database schema. You have a point and click UI that simplifies the design of tables, keys and relationships. There is also a SQL pane in which you can create any PostgreSQL items you want such as functions or triggers. These can then be used from the UI to link functions, views and triggers to your schema.

The Remote Schema and Events tabs are powerful features which I want to cover in more depth in a follow-up article. The Remote Schema tab allows you to set up and consume one or more URL’s as part of your Hasura Schema. This means that you can write a Serverless function, for example, that accepts data to pass to a REST API but which exposes the results as GraphQL. The Events tab allow you to hook into schema-based events and react to them. So, again, you can use Serverless functions to process a new entry in a table and push the results to an endpoint or a different table.

After working through the tutorials I could immediately see how good this software could be. I still had questions around security and manageability:

  • Security - How easy is it to secure and what about Role Base permissions. Firstly, you can secure the entire Hasura instance with a password which is passed in all requests as a request header. It also then means if you try to visit the instance from your browser you will also need the password. Hasura provides full Role-Based permissions that can be added to each table and right down to actions on the table.
  • Manageability - First off migrations. They provide a migration framework through the Hasura console app that means you can push changes from your development instance through to staging and production with a simple command. I have opted to use Digital Ocean as a host as Hasura offered one-click deployment but they also support Azure and Google Could. Applying a custom domain was as easy as creating an A record for your domain and using the IP address of your instance. They also provide a health monitoring endpoint on each instance via /healthz.

All the features I have outlined in this post are available on the free tire and the performance is only limited by your hosting provider. I really am impressed by how we have used Hasura so far and how easy it has made creating API’s.

I will follow up to this post with more detailed howto around the Remote Schema and Event functionality. In the meantime if you have used Hasura for a project or have any interesting tips then let me know via twitter or email.