How to Get, Parse, Query, and Return JSON

 

JSON (JavaScript Object Notation) is a great way to pass data around the Web. It is easy for humans to read and it’s easy to work with in most programming language (including Google Apps Script).

This post will show you a number of useful examples of how to work with JSON with Google Apps Script.

 

(re)introducing JSON

Douglas Crockford Proposed JSONJSON refers to a syntax for organizing information in text, a MIME type, and a file type. They are all part of one big idea from Douglas Crockford (the guy in the picture).

JSON Syntax has two structures: objects, key-value pairs; and arrays, lists of values. The values can be strings, numbers, true, false, null, arrays or objects. For more on the JSON syntax, check out json.org.

The JSON MIME or Media Type, is expressed as application/json for the HTTP Content-Type header. You might see it’s more opinionated, but equivalently parseable brother, application/vnd.api+json.

The JSON file type always has the extension .json and is essentially a text file that organizes information in the syntax mentioned above.

You may have heard people refer to pretty-printed JSON. This term describes JSON text when it is written to a file or printed to a screen so that it is human readable. Pretty-printed JSON typically uses a lot of white space and displays one value per line. JSON is not always pretty printed because these additional spaces and linebreaks slightly increase the size of the file.

Now let’s get some JSON and make something happen!

 

Getting JSON with Google Apps Script

Getting JSON into your Google Apps Script code is just like getting JSON into any other code, you either access it from your file system (in this case Google Drive) or retrieve it from the internet.

 

Downloading JSON from an API with UrlFetchApp

APIs are probably the most entertaining way to get JSON. For this example, we will use the Chuck Norris Quote API. Calls to the http://api.icndb.com/jokes/random API endpoint return JSON that looks like this:

(This JSON is pretty printed but the API does not actually return pretty-printed JSON)

To get request JSON from this API, all you have to do is:

We will see how to parse and query this later.

 

Receiving JSON with doPost

If you want to POST JSON from your computer or another cloud application, you can use the Google Spreadsheet Web App feature to receive the data. From there, it is quite easy to save that JSON text to a file in Google Drive.

The doPost() function runs anytime the web app receives a POST request. It takes one argument. That argument can be named anything you want but it is an object that stores information about the request. In this case, we access the string representation of the JSON data from the request parameter.

To test this little script, you can use Postman to send a POST request with raw JSON data. Or you can copy this into your (Mac) terminal.

When the above POST request is made, a file named post.json will be created in Google Drive with the contents:

 

Accessing JSON Files From Google Drive

Let’s continue from the example above. If you wanted to access the file called post.json from Google Drive,  you can then read the file in as JSON and work with its contents in your script.

 

Parsing and Querying JSON with Google Apps Script

If you have any experience with Javascript, this should be a walk in the park. But in case you are new to it, let’s take a look at the JSON Google Apps Script Class.

The JSON Class offers two different methods for working with JSON. JSON.parse() reads in the JSON text into a Javascript object, and JSON.stringify() serializes (a.k.a. converts) Javascript objects into JSON text strings.

 

Parsing JSON with JSON.parse

As we saw above, when you read in or receive JSON, it will be represented as a string. But in order to use that within your script, you will need to convert it into an object. That is really easy to do. Let’s use the Chuck Norris JSON to examine this.

Now we have Javascript object that we can use in our script.

 

Querying JSON

Querying just means looking something within a data source. In our case, we are less querying than JSON than a Javascript object, but you get the idea. Following from the example above:

Notice, that with Javascript you can access object properties with either “dot notation” or “bracket notation.” Dot notation is convenient but bracket notation is useful when you need to access an object’s property with a variable rather than a string.

Arrays can only be accessed with bracket notation. Note that the index for the array starts with zero and if an array doesn’t have a specified index or an object doesn’t have a specified property, the result will be undefined. JSON uses null, but undefined is the Javascript equivalent.

 

Creating JSON with Google Apps Script

Now that we have seen a few ways that we can get JSON, let’s see a few ways that we can create it. When you have a Javascript object, serializing JSON is very easy:

To provide a user or another application we can reverse the methods that we used to get JSON above.

 

Return JSON with doGet

doGet, similar to doPost, responds to requests to your web app. To return JSON to a GET request, we simply handle the request with doGet, then return JSON text with the ContentService.

The ContentService class provides a set of utilities to create and define several types of output, including images, HTML, and of course, JSON. Whatever the doGet function returns is what is returned the client that makes the request.

 

Send JSON with UrlFetchApp

Earlier, we saw how to receive POST requests and accept the JSON payload data. Here we see how to send the POST request using the Google Apps Script UrlFetchApp function.

The second argument for UrlFetchApp.fetch is an object the specifies the details of the request. In this example, we specified a POST requests with the JSON content-type header and the JSON data from the example above.

 

Save a JSON file to Google Drive

This was show above, but since you’re here, let’s look at it again. To save a JSON file to Google Drive, serialize the Javascript object to text, and use the DriveApp class to save the file to Drive. Again, we will use the JSON from myDog object above.

Yes, it is that simple. Just search for “rhino.json” in Google Drive and you will see a file with that name. You can also dynamically name the file by passing in a variable to the first argument to DriveApp.createFile.

 

Printing a JSON data to Google Spreadsheets

One of the best things about Google Apps Script and Google Spreadsheets is that it is so easy to get data from APIs into spreadsheets. Here’s an example to create a spreadsheet header with the object keys and set the row values to the values of the object.

If you have an array of multiple objects, you could run them all through a loop to populate the contents array and then write the values of all the objects to the spreadsheet.

{“JSON”: “all the things!”}

I hope this offers you confidence to work with JSON in your Google Apps Script future. If I missed anything or any of the examples or instructions were unclear, please let me know in the comments. Also, feel free to share your JSON project in the comments! Happy scripting!

February 7, 2017

Posted In: Programming & Data

Tags: , , ,

One Comment

doGet and doPost Tutorial + 6 Web App Examples

Google Apps Script Web AppsIf you are new to Google Apps Script or programming in general, Google Apps Script Web Apps may seem daunting. The good news is that it is easy to understand with a little trial and error.

This tutorial will give you 12 recipes to help you get started with the doGet and doPost functions and show you how to return HTML, JSON, and plain text. From there, you can go on to customize your Web App to your needs.  But first, let’s get the basics out of the way.

 

What do doGet and doPost do?

Since you are starting to build a web app, you should probably know about HTTP. HTTP is the specification for how different machines on the web send and receive information from one another. HTTP allows several types of “requests,” but what we see most often are GET and POST requests. doGet and doPost functions define how a Google Apps Script web app scripts handle GET and POST requests from other machines and determine what they return or do.

A simple example of a GET request happens every time you enter a URL into your web browser. The web server at that domain then handles your request and returns an HTML web page. GET requests are the most common type of request on the internet. For our purposes, GET requests are made of up of a domain, e.g., example.com a path, e.g., /fruits/apples/ and often a query string e.g.,?variety=fuji&count=4With Google Apps Script, the domain will always be script.google.com, and the path will always look with something like /macros/s/AKf...ycb/exec (or /dev), and the query string can be whatever you want it to be, in order to specify how the web app dynamically generates the content it returns.

POST requests are similar to GET requests but slightly more powerful. Rather than asking for content, they are used to send data from one machine to another. The most common use of POST requests is web forms, like signup forms and ecommerce checkout pages. POST requests are not limited to sending form text though. They can transfer image, audio, and a number of other types of files. This tutorial will only cover JSON, but if you need more than that, you probably don’t need this tutorial!

 

Your First Web App “Hello World”

Let’s get a few easy examples out of the way. The next three code snippets serve essentially the same content in three different formats.

To try them out:

  1. Copy and paste them into a new Google Apps Script.
  2. Click Publish on the main menu and then Deploy as web app…
  3.  Click “Test web app for your latest code.”

 

Returning Plain Text

The ContentService.createTextOutput method is the most basic way to return content from a GET request but is the building block for most other types of content. The TextOutput objects created by this method returns a response of type, text/plain by default, but can be set to ATOM, CSV, ICAL, JAVASCRIPT, JSON, RSS, TEXT, VCARD, and XML.

 

Returning JSON

This is a simple pattern. Just serialize the Javascript object as JSON using JSON.stringify(), and set the content-type header with the setMimeType() method. This is important when returning different media types because you don’t want to let the recipient of your request guess the MIME type. For more on this, see my Google Apps script JSON REST API tutorial.

Returning HTML

The HTMLService.createHtmlOutput() method has two purposes: to clean of any security risks before the HTML is returned to the browser and to implicitly set the MIME type to text/html. To construct web pages, you can assemble the HTML string, as shown below, or you can use templates.

 

Dynamic Web Pages with the HTML Service

 

External Content, Query Strings, and HTML Templates

Let’s make this interesting. Google Apps Script can assemble web pages based on dynamic inputs. In the example below, we use an external CSS file and the query string parameters to create dynamic content. The doGet and doPost parameter take one default argument (commonly e), which holds information about the request “event.” From that e object, you can pull of information like query string values, as shown below. See the working version of the web app here.

To take this one step further, you can create HTML template files in Google Apps Script and evaluate those templates rather than creating the template in the script. Additionally, you can use information from the Google Drive, Gmail, and other Google APIs to populate your app.

 

Google Apps Script doPost Example

 

An API Proxy

This example was inspired by my last post about custom Slack slash commands. This doPost example demonstrates how to handle POST requests, access request query string parameters, make HTTP requests to other services (in this case, the Chuck Norris joke API) with UrlFetchApp, and break the script into functions.

The reason I made this was because I got data from the Slack app that was formatted as a query string,

that I needed to use to get data from an API that would be formatted as JSON,

to then send back to Slack app as plain text:

This problem also exists if you want to use another service that requires that data be formatted in a specific way- like the IFTTT Maker Channel.

 

More than Just Returning Content

 

Dynamically Create and Save a PDF File to Google Drive

This example is similar to the dynamic HTML example above but instead of making a dynamic web page, it creates and saves a PDF document in Google Drive. This would also work with doPost in a situation where you would want to send more information to the app when creating you PDF. Check out the live script. *Note, this will create a PDF file in you Google Drive each time you go to the URL.

This takes the functionality of the Google Apps Script web app one step further. This script demonstrates the capability to carry out any task or trigger any other service just from GET’ing or POST’ing to your web app.

 

 

Publishing and Permissions

When you are ready to launch your new Google Apps Script web app. You should save a version and define who and how users and other machines can access your app. There are a couple important things to note.

If your app offers any capability to interact with other Google Drive services, the app should be set to Execute the App As:  User accessing the web app.

If you want to allow other services, to interact with your web app (as an API or proxy, the app should be accessible to Execute the App As: Me (myemail@gmail.com) and allow Anyone, even anonymous.

 

I hope this gets you started! Let me know if you are having any trouble getting your app up and running in the comments. Happy hacking!

January 30, 2017

Posted In: Web App

Tags: , , ,

Leave a Comment