Converting text to speech with AWS Polly

I wanted to try my hand at using the AWS Polly text-to-speech service.  Polly offers several different voices and supports multiple languages, most of which sound pretty good, especially if you use SSML when passing text.  SSML is where the character of the speech (rate, tone, pitch, etc) come into play.  See here for more detail.

What I’ve done is created a script to interact with the AWS Polly API using PHP and store the output into an S3 bucket.  Click here to try it out.

Step 1: Creating the IAM User

This has been outlined in many prior posts so I won’t go into detail.  We’ll be using the AmazonS3FullAccess and AmazonPollyFullAccess permission policies (screenshot of my user summary here).  If you don’t plan on saving the results to S3, you don’t need the S3 policy attached (obviously).

Step 2:  Converting our text to speech

<?php
require '/home/bitnami/vendor/autoload.php'; 

//Prep the Polly client and plug in our IAM credentials
use Aws\Polly\PollyClient;
$clientPolly = new PollyClient([
'version' => 'latest',
'region' => 'us-west-2', //I have all of my AWS stuff in USW2 but it's merely preference given my location.
'credentials' => [
  'key' => '', //IAM user key
  'secret' => '', //IAM user secret
 ]]);

//Configure our inputs and desired output details
$resultPolly = $clientPolly->synthesizeSpeech([
'OutputFormat' => 'mp3',
'Text' => "<speak>This is my example.</speak>",
'TextType' => 'ssml',
'VoiceId' => 'Joey', //A list of voices is available here: https://docs.aws.amazon.com/polly/latest/dg/voicelist.html
]);

//Get our results
$resultPollyData = $resultPolly->get('AudioStream')->getContents();

?>

Step 3: Saving the MP3 file to S3

<?php
//Set timezone to use in the file name
date_default_timezone_set('America/Los_Angeles');

//Prep S3 client and plug in the IAM credentials
$clientS3 = new Aws\S3\S3Client([
'version' => 'latest',
'region' => 'us-west-2', //Region of your S3 bucket
'credentials' => [
  'key' => '', //Same IAM user as above
  'secret' => '',  //Same as above
 ]]);

//Put the Polly MP3 file to S3
$resultS3 = $clientS3->putObject([
  'Key'         => date("Y-m-d_H:i:s").'.mp3',
  'ACL'         => 'public-read',
  'Body'        => $resultPollyData,
  'Bucket'      => 'fkpolly',
  'ContentType' => 'audio/mpeg',
  'SampleRate'  => '22050'
]);

//Return a link to the file
echo "<a href=\"".$resultS3['ObjectURL']."\">Listen</a>";
?>

Try It Out

I’ve saved my example for ongoing tinkering.  Click here to try it out.

WordPress Plugin Recommendations – 2018 Edition

I’m not an optimization expert nor am I a WP power user but I have been using the platform for over ten years.  I have a strong preference for plugins that are lightweight, easy-to-implement and configure, and have a clean removal (plugins which leave artifacts are a huge pet peeve of mine).  Here’s a list of my must-have plugins for almost all WordPress installations.

WordFence

My complaints with WordFence surround it’s initially annoying push for upgrading to the premium version. You can dismiss/hide those, though, which leaves you with a pretty effective solution at thwarting most low-end abusive crawlers/sniffers. The highlight is the threshold with auto-block feature which allows you to block traffic if activity breaches certain thresholds.  It just makes things easy.

Wordfence Security – Firewall & Malware Scan

Instant Images

Although not my area of interest, it’s handy.  Instant Images pulls free-to-use (under the CC0 license) images from UnSplash directly into your WordPress media library.  It saves a few clicks and makes things easier when in need for stock images.

Instant Images – One Click Unsplash Uploads

Velocity

Velocity is a nifty plugin that allows you to embed YouTube/Vimeo/SoundCloud media without loading the heavy iframes/JS libraries until the user engages with the media.  This saves a ton on overall load time/size.  It also allows you to set a custom preview image for your embedded media so that’s a small plus.  Using this plugin, I reduced my page load size from 1.73mb to 634kb and reduced the number of requests onload from 89 to 59 — this is a 1 second decrease in pageload for devices on 3G networks!

Velocity – WordPress Lazy Load for Video and Audio

commonWP

commonWP is a plugin which uses the jsDelivr CDN for common WordPress JS files.  It’s super easy and low-risk to implement thanks to really well thought out work from the creator.  I hope this expands to CSS files in the near future.

commonWP

Enlighter

Enlighter is an easy-to-use syntax highlighter that supports most languages.  While there are several syntax highlighters out there, I like this one in particular for it’s easily modifiable CSS and clean editor integration.

Enlighter – Customizable Syntax Highlighter

WP Mail SMTP

As the name suggests, this plugin enables easy use of SMTP for mail on your WordPress installation.  No need to go deeper than your WP-Admin pages to configure mail for your WordPress installation.  The downside is that it was recently acquired by WPForms so I’m guessing it’ll be turned into an intrusive, premium-hocking version in the near future.

WP Mail SMTP by WPForms

Using Natural Language Processing (AWS Comprehend) to Analyze Text (Cardi B Lyrics)

Humans spend a lot of time reading, analyzing, and responding through text (emails, chats, etc). A lot of this is inefficient or not for pleasure (such as the amount of payroll companies spend to read through feedback emails or the amount of time I spend sifting through Outlook each day). Using Natural Language Processing (NLP), we can reduce the inefficient and not-for-pleasure reading we do so that time can be re-invested into something more productive or fulfilling.

For fun, I scrapily ran the lyrics to Cardi B’s “I Like It” through AWS Comprehend to see what its response would be. I also ran a review of Mission Impossible: Fallout through the same service.  The full output for Cardi B can be viewed here.  The full output for Mission Impossible: Fallout can be viewed here.

While these are low value examples, a more real-world use case for Comprehend could be using AWS Comprehend to detect the language of emails sent to your company to adjust the routing destination in real-time (should they go to you English team or your Spanish?). Another example would be using Comprehend to collect feedback on your new product launch or ad campaign. For example, we could easily capture Twitter mentions for a brand, funnel those into an S3 bucket, and run the contents of that buck to split out negative vs positive vs neutral vs mixed sentiment mentions. From there, we could surface the most frequent adjectives and entities mentioned for each group of each sentiment bucket. It’s a cheap, quick way to customer capture and analyze feedback that would otherwise be ignored.

Initial Setup
In each of the last couple of posts, I’ve outline how to create an IAM user for your project so I won’t repeat that again.  After we have our IAM user created and the credentials added to our /.aws/credentials file, we’ll import the AWS PHP SDK and ComprehendClient.  Next, we’ll create a Comprehend client, define the API version, region, and credentials profile to use:

<?php

require '/home/vendor/autoload.php'; 
use Aws\Comprehend\ComprehendClient;

$client = new Aws\Comprehend\ComprehendClient([
    'version'     => '2017-11-27',
    'region'      => 'us-west-2',
    'profile'     => 'fk-comprehend',
]);
$review="Your Comprehend text";

Exploring the Options
In this example, we’ll detect the language(s), entities (objects, businesses, etc), key phrases, sentiment, and syntax (parts of speech) of our sample texts (Cardi B lyrics and a movie review).  For all of these except DetectDominantLanguage, the language is a required input.  If we use Comprehend to identify that first, then we can simply repeat its output in later functions.  For each output, Comprehend also spits out a confidence score which basically  tells you how confident it is in the output.  This could be used to ignore low-confidence suggestions, thus increasing the accuracy of the models you build using Comprehend.

DetectDominantLanguage Example
This will detect the language and spit out the ISO abbreviation.

//Detecting Dominant Language
$result = $client->detectDominantLanguage([
    "Text" => "$review",
]);

echo "<h1>DetectDominantLanguage</h1><pre>";
print_r($result);
echo "</pre>";

foreach ($result['Languages'] as $phrase) {
    echo "Language ".$phrase['LanguageCode']." has a confidence score of ".round($phrase['Score']*100)."%.<br />";
}

DetectSentiment Example

//Detecting Sentiment
$result = $client->detectSentiment([
    "LanguageCode" => "en",
    "Text" => "$review",
]);

echo "<h1>DetectSentiment</h1><pre>";
print_r($result);
echo "</pre>";

echo "Sentiment: ".$result['Sentiment']."<br />";
echo "Positive: ".round($result['SentimentScore']['Positive']*100)."%<br />";
echo "Negative: ".round($result['SentimentScore']['Negative']*100)."%<br />";
echo "Neutral: ".round($result['SentimentScore']['Neutral']*100)."%<br />";
echo "Mixed: ".round($result['SentimentScore']['Mixed']*100)."%<br />";

DetectKeyPhrases Example

//Detecting KeyPhrases
$result = $client->detectKeyPhrases([
    "LanguageCode" => "en",
    "Text" => "$review",
]);

echo "<h1>DetectKeyPhrases</h1><pre>";
print_r($result);
echo "</pre>";

foreach ($result['KeyPhrases'] as $phrase) {
    echo "Phrase ".$phrase['Text']." has a score of ".round($phrase['Score']*100)."%.<br />";
}

DetectSyntax Example

//Detecting Syntax
$result = $client->detectSyntax([
    "LanguageCode" => "en",
    "Text" => "$review",
]);

echo "<h1>DetectSyntax</h1><pre>";
print_r($result);
echo "</pre>";

foreach ($result['SyntaxTokens'] as $syntax) {
    echo "Phrase ".$syntax['Text']." is as ".$syntax['PartOfSpeech']['Tag']." (with ".round($syntax['PartOfSpeech']['Score']*100)."% confidence).<br />";
}

DetectEntities Example

//Detecting Entities
$result = $client->detectEntities([
    "LanguageCode" => "en",
    "Text" => "$review",
]);

echo "<h1>DetectEntities</h1><pre>";
print_r($result);
echo "</pre>";

foreach ($result['Entities'] as $syntax) {
    echo "Phrase ".$syntax['Text']." is as ".$syntax['Type']." (".round($syntax['Score']*100)."% confidence).<br />";
}

The Results for Cardi B and Tom Cruise

The full output for Cardi B can be viewed here.  This one is the most interesting of the two as “I like it” has a Spanish verse.  You can see how Comprehend dealt with it when it was passed as English.  It also does a good job of determining when “bitch” is a noun vs an adjective except in the line “Where’s my pen? Bitch I’m signin'” — I’m unsure as to why.

The full output for Mission Impossible: Fallout can be viewed here.  The interesting piece here is the sentiment analysis: NEUTRAL (8% positive, 36% negative, 39% neutral, and 17% mixed).  After reading the review, I would say this is pretty in-line with the reviewer and Comprehend did a good job of identifying the overall sentiment of the article.

Automatically Creating Lightsail Instance Snapshots

Given the target audience of Lightsail, I would expect UI-based functionality for automating snapshots and other common tasks; however, this doesn’t exist.  Creating snapshots is an important task – I create snapshots before I make any major changes and every few days.  In the event I screw something up or if something happens to my instance, I can simply spin up a new instance from an old snapshot – no big deal.

In addition to the lack of UI-based functionality, the default IAM policies don’t apply to Lightsail, either.  Given the age of Lightsail, I would think this would be built into IAM default policies by this point.

In the guide below, we’ll:

  1. Create an IAM policy to manage our Lightsail snapshots
  2. Create an IAM user to use that IAM policy
  3. Add our IAM user to our AWS credentials file
  4. Create a Lightsail snapshot using the AWS CLI

Beyond creating snapshots, there AWS CLI offers all commands needed to manage Lightsail – I encourage you to explore: https://docs.aws.amazon.com/cli/latest/reference/lightsail/index.html

Create the needed IAM Policy

  1. From the IAM page of the AWS Console, select Policies.
  2. From there, click “Create Policy” and select the json tab.  We’ll use this policy which will limit actions to just the creation and listing of instance snapshots:
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": [
                    "lightsail:GetInstanceSnapshot",
                    "lightsail:GetInstanceSnapshots",
                    "lightsail:CreateInstanceSnapshot"
                ],
                "Resource": "*"
            }
        ]
    }

    Here’s a screenshot of the inputs.

  3. Click “Review Policy” and then we’ll give it a name (I’ve used “LightsailSnapshotCreate”) and then click “Create Policy”

Create the needed IAM User

  1. Back on the IAM page of the AWS Console, click on Users.
  2. We’ll name the user (I’ve used “FKSnapshotCreate” and check the “Programmatic Access” box.  Screenshot.
  3. On next page, we’ll attach the policy we created (“LightsailSnapshotCreate”) and create the user.  Screenshot.
  4. Lastly, we’ll copy the access ID and key into our credentials file.  Screenshot.

Adding the IAM User Access Key/Access ID to our Credentials file

  1. Open your credentials file and add in another user (as described here).
  2. In my project, I have a user named “fk-createsnapshot” so my credentials file looks something like this.

Creating a snapshot using AWS CLI

  1. Now that we have everything configured, we just need to run the command.  We’ll do this with the following command:
    aws lightsail create-instance-snapshot --instance-name FigarosKingdomWP --instance-snapshot-name FK-2018-09
    -09 --profile fk-createsnapshot --region us-west-2

    “–instance-name FigarosKingdomWP” – this is the name of the instance from the Lightsail console that you’re wanting to snapshot.
    “–instance-snapshot-name FK-2018-09-09” – this is the name of the snapshot.  It can be anything you like.
    “–profile fk-createsnapshot” – this is the IAM User (same one we created above) that we want to use with this command.
    “–region us-west-2” – this is the region of the instance.

  2. Once executed, you should see output similar to this:
  3. Browse to the Lightsail Console and you should see your new snapshot: screenshot.

Creating a shell script to automate snapshots

We’ll create a shell script (I’m using fk-createsnapshot.sh) and add in our command.  I’ve also added variables for the date so that the snapshot name matches the date.  You can find more about this here.  Here’s my fk-createsnapshot.sh:

#!/bin/bash 
aws lightsail create-instance-snapshot --instance-name FigarosKingdomWP --instance-snapshot-name FK-$(date +%Y-%m-%d) --profile fk-createsnapshot --region us-west-2

Adding a cron job to run the shell script

In our crontab (crontab -e), we’ll set it to run at 3am every day (adjust to your script location):

0 3 * * * /home/bitnami/fk-createsnapshot.sh

It may be a good idea to set it to run sooner just to verify that everything is working as intended.  Otherwise, you’ll start seeing new snapshots appear in your Lightsail console every day at 3am!

Using this same approach, you can take pretty much all actions for your Lightsail instances directly through the CLI.  Check out the AWS CLI documentation for more.

 

Using AWS Lambda to Send SNS Topics in CloudWatch

AWS Lambda enables you to run code without managing a server.  You simply plop in your code and it does the rest (no maintenance, scaling concerns, etc).  The cost is only $0.20 per 1 million requests/month and the first million requests are free each month.

In the previous post, I setup an SNS topic. I’m extending this further so that a node.js function will be triggered in AWS Lambda each time my SNS topic is triggered. This Lambda function will feed metrics into AWS CloudWatch which will allow me to chart/monitor/set alarms against events or patterns with my SNS topic.  A practical use case for this could be understanding event patterns or logging SNS messages (and their contents) sent to your customers.

Creating your Lambda Function

From the Lambda page of the AWS console, select “Create Function”.  From here, we’ll author from scratch.  Below are the inputs I’ve used for this example:
Name: SNSPingerToCloudWatch
Runtime: Node.js 8.10
Role: Choose and existing role
Existing role: lambda_basic_execution

On the page after selecting “Create Function”, we’ll click “SNS” from the “Add Triggers” section and then select our SNS topic in the “Configure Triggers” section.  Then click “Add” and “Save”.  Here’s a screenshot of the final state.

Next, click on your function name (SNSPingerToCloudWatch) in the flow chart and scroll to edit the function code.
The JS we’ll use:

exports.handler = async (event, context) => {
    const message = event.Records[0].Sns.Message;
    console.log('Pinger says:', message);
    return message;
};

Under Basic Settings, I’ve set the timeout duration to 5 seconds (because that’s the timeout duration I have set in my SNS topic PHP script) You can add descriptions, throttles, etc but I’m leaving those at the defaults.  Here’s a screenshot of my final config for this Lambda function.

Once complete, click “Save” again and then we’re ready to test. I manually fired my SNS topic and jumped over to the “Monitoring” tab of the console. It took a minute or so but I saw my event appear. From here, you can view the log details in CloudWatch, as well.