3 minutes reading time
Nowadays developing AIs requires data, and lots of it. Emotion recognition is no different. Training an AI to recognition requires training on facial data. If you don't have your own dataset, there are public sets but many are licensed for researcher only.
Kaggle seems to have a usable one
Their license seems permissive
The Licensor grants to You a worldwide, royalty-free, non-exclusive, perpetual, irrevocable copyright license to do any act that is restricted by copyright over anything within the Contents, whether in the original medium or any other. These rights explicitly include commercial use, and do not exclude any field of endeavour.
The database gives us test and training faces. Low rez, but cropped and classified as: angry, disgust, fear, happy, neutral, sad, and surprised
Rather than training my own set, I've decided to try out a couple of existing recognition APIs.
the API is very straightforward. Get an API token, pass some JSON, POST you token and a set of files to it, read it back out
python3 -m venv luxand
. luxand/bin/activate
Like many APIs you can do all your work with the Python requests module and an API key from
pip3 install requests
Get an account at Luxand. I used my codepala email to setup an account.
After registering I got an API token Cid38vd932hfadie4rhc33csrhdeZsie
import requests
import json
API_TOKEN = "Cid38vd932hfadie4rhc33csrhdeZsie"
def emotions( image_path ):
url = "https://api.luxand.cloud/photo/emotions"
headers = { "token" : API_TOKEN }
files = { "photo": open( image_path, "rb")}
response = requests.post(url, headers=headers, files=files)
result = json.loads(response.text)
if response.status_code == 200:
return response.json()
else:
print("Can't recognize people:", response.text)
return None
#image_path = "./PublicTest_63454133.jpg"
#image_path = "./PublicTest_96013755.jpg"
image_path= "a6bfokalx1xc1.jpeg"
#image_path= "./moeka-hoshi-v0-qn6ya2421auc1.jpg"
result = emotions(image_path)
print(result)
Sadly it seems to fail about half of the images I sent with
(luxand) ggallard@BOOMSTICK:/mnt/d/Dev/Emo/luxand_tests$ python3 test1.py
{'status': 'failure', 'message': "Couldn't recognize emotions in the photo"}
It did detect some faces though, like the cast of Shogun
(luxand) ggallard@BOOMSTICK:/mnt/d/Dev/Emo/luxand_tests$ python3 test1.py
{'status': 'success', 'faces': [{'dominant_emotion': 'fear', 'emotion': {'angry': 0.007313171371199131, 'disgust': 5.872549241264984e-11, 'fear': 99.82080457458551, 'happy': 0.03940112543603783, 'neutral': 0.03498433342016738, 'sad': 0.06902363624891773, 'surprise': 0.0284793937069945}, 'region': {'h': 55, 'w': 55, 'x': 1072, 'y': 173}}, {'dominant_emotion': 'happy', 'emotion': {'angry': 0.0029466529667843133, 'disgust': 4.42003961209636e-08, 'fear': 9.040934969561931e-05, 'happy': 93.23827028274536, 'neutral': 6.752225011587143, 'sad': 0.002435347778373398, 'surprise': 0.004030428681289777}, 'region': {'h': 238, 'w': 238, 'x': 894, 'y': 231}}, {'dominant_emotion': 'angry', 'emotion': {'angry': 41.76114499568939, 'disgust': 3.864876774173354e-06, 'fear': 0.004789933882420883, 'happy': 0.007058105256874114, 'neutral': 30.795982480049133, 'sad': 27.43098735809326, 'surprise': 2.9014981350883318e-05}, 'region': {'h': 112, 'w': 112, 'x': 105, 'y': 365}}]}
(luxand) ggallard@BOOMSTICK:/mnt/d/Dev/Emo/luxand_tests$
From aws
Amazon Rekognition is a cloud-based image and video analysis service that makes it easy to add advanced computer vision capabilities to your applications. The service is powered by proven deep learning technology and it requires no machine learning expertise to use. Amazon Rekognition includes a simple, easy-to-use API that can quickly analyze any image or video file that’s stored in Amazon S3.
Basically we need to upload images to S3 bucket then call Rekognition on the bucket
python3 -m venv rekogenv
pip3 install boto3
arn: arn:aws:s3:::facex-rekog-img-ciefuxleirrhxsr
Create Policy basically this (but I used the web interface):{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::facex-rekog-img-ciefuxleirrhxsr",
"arn:aws:s3:::facex-rekog-img-ciefuxleirrhxsr/*"
]
}
]
}
I called it facex_rekog_s3
basically this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rekognition:DetectFaces"
],
"Resource": "*"
}
]
}
Usersfxrekog and added the policies from aboveXXXXXXXXXXXXXXXXXX CiehCdieazce+sdierZXZeVhlSawIehcq2zXier/
import boto3
from botocore.exceptions import NoCredentialsError
def upload_file_to_s3(file_name, bucket, object_name=None):
"""
Upload a file to an S3 bucket
:param file_name: File to upload
:param bucket: Bucket to upload to
:param object_name: S3 object name. If not specified then file_name is used
:return: True if file was uploaded, else False
"""
# If S3 object_name was not specified, use file_name
if object_name is None:
object_name = file_name
# Create an S3 client
s3_client = boto3.client('s3')
try:
# Upload the file
s3_client.upload_file(file_name, bucket, object_name)
print("File uploaded successfully")
return True
except FileNotFoundError:
print("The file was not found")
return False
except NoCredentialsError:
print("Credentials not available")
return False
# Example usage
file_path = 'path/to/your/file.jpg' # Replace with the path to your file
bucket_name = 'your-bucket-name' # Replace with your bucket name
# Call the function
upload_file_to_s3(file_path, bucket_name)
import boto3
def detect_emotions(image_bucket, image_name):
client = boto3.client('rekognition', region_name='us-east-1')
response = client.detect_faces(
Image={
'S3Object': {
'Bucket': image_bucket,
'Name': image_name
}
},
Attributes=['ALL'] # This tells Rekognition to retrieve all attributes, including emotions
)
for faceDetail in response['FaceDetails']:
print('Emotions:')
for emotion in faceDetail['Emotions']:
print(f"{emotion['Type']}: {emotion['Confidence']:.2f}%")
return response['FaceDetails']
# Replace 'your-bucket-name' and 'image-file.jpg' with your S3 bucket and image file
detect_emotions('your-bucket-name', 'image-file.jpg')