Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • M MathJax
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 304
    • Issues 304
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 15
    • Merge requests 15
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • MathJax
  • MathJax
  • Wiki
  • CDN Hosting at Google Cloud Storage

CDN Hosting at Google Cloud Storage · Changes

Page history
re-write to reflect CloudFlare setup and other fixes authored May 14, 2014 by Peter Krautzberger's avatar Peter Krautzberger
Hide whitespace changes
Inline Side-by-side
CDN-Hosting-at-Google-Cloud-Storage.md
View page @ c79bd899
*Note: these are preliminary notes about using Google Cloud Storage as a CDN service* *Note: these are notes about using Google Cloud Storage as a CDN service. We use GCS in combination with CloudFlare which means we can get away with some things..*
### Preliminaries ### Preliminaries
...@@ -10,35 +10,39 @@ ...@@ -10,35 +10,39 @@
Notes: Notes:
* gsutil option: -m multithreaded (higher load+cost, faster) * gsutil option: -m multithreaded (higher load+cost but faster)
* cp options: recursively (-R), compressed (-z), verbose (-v) https://developers.google.com/storage/docs/gsutil/commands/cp * cp options: recursively (-R), compressed (-z), verbose (-v) https://developers.google.com/storage/docs/gsutil/commands/cp
* Note: compressed is necessary to get gzipped delivery. Google Cloud storage does not compress on the fly but should decompress on the fly for old browsers that can't accept decompressed files. * Note: compressed is necessary to get gzipped delivery. Google Cloud storage does not compress on the fly but should decompress on the fly for old browsers that can't accept decompressed files.
gsutil -m cp -Rzv mathjax gs://mathjax-cdn/ gsutil -m cp -Rzv mathjax gs://our-bucket/
This also works within GCS, i.e., bucket to bucket. Note that metadata isn't preserved by default (ACLs etc); use `-p` option to preserve them (*warning*: this incurs higher costs). This also works within GCS, i.e., bucket to bucket.
**Note that metadata isn't preserved by default** (public access, caching, custom headers etc); use `-p` option to preserve them (*warning*: this incurs higher costs).
`gsutil -m cp -Rp gs://bucket1/file1 gs://bucket2/` `gsutil -m cp -Rp gs://bucket1/file1 gs://bucket2/`
(You'll still need to set the CORS for each bucket though; see below.) (You'll still need to set the CORS for each bucket though; see below.)
### make bucket public ### make files public
Notes: Notes:
* should only be needed once
* https://developers.google.com/storage/docs/gsutil/commands/acl#set * https://developers.google.com/storage/docs/gsutil/commands/acl#set
gsutil -m acl set -R -a public-read gs://mathjax-cdn This has to be done whenever files are uploaded (**TODO**: automate!)
gsutil -m acl set -R -a public-read gs://our-bucket/path/to/file_or_wildcard
### set CORS ### set CORS
Notes: Notes:
* should only be needed once * should only be needed once per bucket
* https://developers.google.com/storage/docs/cross-origin * https://developers.google.com/storage/docs/cross-origin
create XML file with Create an XML file with
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<CorsConfig> <CorsConfig>
...@@ -60,8 +64,7 @@ create XML file with ...@@ -60,8 +64,7 @@ create XML file with
Save as `cors.xml` and run Save as `cors.xml` and run
gsutil cors set cors.xml gs://fonts.acmecompany.com/ gsutil cors set cors.xml gs://our-bucket
### mime-type headers ### mime-type headers
...@@ -69,32 +72,23 @@ Notes: ...@@ -69,32 +72,23 @@ Notes:
* https://developers.google.com/storage/docs/gsutil/commands/setmeta * https://developers.google.com/storage/docs/gsutil/commands/setmeta
* the `**` wildcard includes subdirectories, https://developers.google.com/storage/docs/gsutil/addlhelp/WildcardNames * the `**` wildcard includes subdirectories, https://developers.google.com/storage/docs/gsutil/addlhelp/WildcardNames
* EOT and SVG `content-type` headers are detected correctly during upload. * EOT and SVG `content-type` headers are detected correctly during upload to GCS.
* WOFF is set to the old mime type `x-font/woff`; that works but we decided to go with the future-proof `font-woff`. Either change it as below or [see advice here](https://github.com/GoogleCloudPlatform/gsutil/issues/183#issuecomment-35741417). * WOFF is set to the old mime type `x-font/woff`; that works but we decided to go with the future-proof `font-woff`. Either change it as below or [see advice here](https://github.com/GoogleCloudPlatform/gsutil/issues/183#issuecomment-35741417) to do this automatically on upload.
* For otf: * For otf:
gsutil setmeta -r -h "Content-Type:font/opentype" -h "Cache-Control:public, max-age=3600" gs://mathjax-cdn/mathjax/**.otf gsutil setmeta -r -h "Content-Type:font/opentype" gs://our-bucket/mathjax/**.otf
* For otf: * For otf:
gsutil setmeta -r -h "Content-Type:application/font-woff" -h "Cache-Control:public, max-age=3600" gs://mathjax-cdn/mathjax/**.woff gsutil setmeta -r -h "Content-Type:application/font-woff" gs://our-bucket/mathjax/**.woff
### Caching
### caching **Note** Since we are combining GCS with CloudFlare's caching, we will have short caching times on GCS.
**This needs to be updated to reflect the new setup with GCS+CloudFlare -- shorter caches but additional caching on CloudFlare's end** This has to be repeated whenever files get uploaded from local.
This is a one time task. Proposed caching is 3 months for all but `MathJax.js` which gets 1 week. $ gsutil -m setmeta -r -h "Cache-Control:public, max-age=3600" gs://our-bucket/mathjax/
$ gsutil -m setmeta -r -h "Cache-Control:public, max-age=7776000" gs://mathjax-cdn/mathjax/
$ gsutil -m setmeta -r -h "Cache-Control:public, max-age=3600" gs://mathjax-cdn/mathjax/**/MathJax.js
This is theoretically risky. Since caching can only be specified after making the file public, it could theoretically happen that MathJax.js is called and cached before the second command runs -- fixing it to a 3 month caching that cannot be undone.
Well, it didn't happen. So we're good.
### Logging
Another one time task. Instructions are at https://developers.google.com/storage/docs/gsutil/commands/logging
### CNAME ### CNAME
...@@ -102,11 +96,16 @@ To use a CNAME for a bucket, the bucket needs to be named like the CNAME, see ht ...@@ -102,11 +96,16 @@ To use a CNAME for a bucket, the bucket needs to be named like the CNAME, see ht
You also need to verify domain ownership; see https://support.google.com/webmasters/answer/35179?hl=en. You also need to verify domain ownership; see https://support.google.com/webmasters/answer/35179?hl=en.
## TODO ## Testing
* CORS allow-origin [vital]
* A simple test: switch mathjax.org over to the GCS origin (bucket.storage.googlapis.com) and use saucelabs to check rendering across IE, FF, Chrome, Safari.
* Since CORS is set on bucket level, we should not have to worry about it.
* font mimetypes [non-vital]
* **TODO** write script to `curl -I` all woff and otf files, compare header.
### Logging
We don't log on GCS anymore since we combine it with CloudFlare (and get their stats).
* tests. For the record, instructions are at https://developers.google.com/storage/docs/gsutil/commands/logging
* CORS [vital]
* A simple test: switch mathjax.org over to the origin. user browsershots/browserstack etc to check rendering across IE, FF, Chrome, Safari.
* mimetype [non-vital]
* write script to `curl -I` all woff and otf files, compare header.
* document log analysis
Clone repository

MathJax Wiki

  • Contributing
    • Contributor License Agreement etc
    • Quick guide to translating mathjax
  • Development
    • Development Process
      • Release Process Checklist
      • Documentation Update Process
      • Source Control Policies
    • Design Documents
      • MathJax Roadmap
      • CDN Hosting
        • Managing Rackspace Cloud Files & CDN
        • Directory Structure
        • .htaccess settings
        • CDN requirements
        • CDN Hosting at Google Cloud Storage
      • Performance Discussion
      • Profiling and Diagnostics Tools
      • Configuration Options
      • Documentation generation guide
      • Testing
        • Platforms supported
        • Test Machines
  • MathJax web presence
  • Drafts