I've had this automation running for about a month, and its hilarious. Amuses every visitor I get, and knocks down my ego a few pegs every time I come home. You can of course adjust the "spice level" of the responses by tweaking the instructions prompts.
alias: Person Detection - Front
description: ""
triggers:
entity_id:
- binary_sensor.driveway_person_detected
to: "on"
for:
hours: 0
minutes: 0
seconds: 3
entity_id:
- binary_sensor.porch_motion_sensor_motion_detection
to: "on"
for:
hours: 0
minutes: 0
seconds: 2
conditions: []
actions:
device_id: 53b3ef12acc92505d47e0f628ab40031
entity_id: ea7f14f1cedc0194bd7bcf97923f08b7
domain: light
brightness_pct: 100
- action: llmvision.image_analyzer
metadata: {}
data:
remember: true
use_memory: false
include_filename: false
target_width: 1280
max_tokens: 100
temperature: 0.8
generate_title: true
expose_images: true
provider: 01JQM2QBVXMZAGNZ3SMW63A8YB
image_entity:
message: >-
About You
You perform detailed security camera image analysis at almost a forensic
level. You will be shown one or more still frames from multiple security
cameras facing the driveway of a house.
Analysis Instructions
If you see a person or a new car (especially on the road) in the frame,
describe the person or car in a single sentence, use crass humor. Here
is an example: "At the front of the house there is a white delivery
truck (the kind kidnappers use) and another pedo looking (you know the type) person
wearing a tasteless orange vest carrying a large purple box". This is
just an example, be creative, and brief.
Don't describe what the individual camera is showing, only the complete
scene where a visitor is coming by the house. Don't say first image, 2nd
image, analyze both as a single thing. Don't focus on the surroundings,
focus on one or two specific details about the person from the image,
and be brief. People can be detected on the edges of the images, so look
carefully. Avoid saying "rolled out of bed" or "looks like he owns the
place" as you seem to say that a lot, be very specific and creative in
your description.
If you don't see people or new cars, say something like "Something is
happening out front, but I'm not sure what exactly" (feel free to
rephrase, just keep it short).
Things to ignore
There are usually 2 Teslas parked in the bottom of the frame, so don't
mention those unless one is missing. There may also be a white truck or
car in the top left corner of the frame, these are neighbor’s cars and
this is normal.
model: gpt-4o
response_variable: person_description
- action: media_player.volume_set
metadata: {}
data:
volume_level: 0.7
target:
entity_id:
-
media_player.voice_box
-
action: tts.speak
metadata: {}
data:
cache: true
media_player_entity_id: media_player.voice_box
message: "{{ person_description.response_text }}"
target:
entity_id: tts.elevenlabs
-
wait_for_trigger:
-
trigger: state
entity_id:
- binary_sensor.qolsys_panel_front_door
to: "on"
for:
hours: 0
minutes: 0
seconds: 3
timeout:
hours: 0
minutes: 3
seconds: 0
milliseconds: 0
continue_on_timeout: false
- action: google_generative_ai_conversation.generate_content
metadata: {}
data:
prompt: >-
There is a person(s) entering the front door of a house, say a very
personalized greeting that will play over the PA system to this person
right after they enter. Do a short (2-3 sentences max), not politically
correct and funny roast (you can use profanity, as its all in good fun)
based on the description of the person and event. Dont say anything else
except the short roast directed at them, and be random as this prompt
will be used many times. Be sure to focus on the specifics of the
persons description, and always mention them all in your roast. Then
welcome them into the house and suggest a task or activity they can do.
Be creative and controversial.
The camera detected the following event:
{{person_description.response_text}}
response_variable: visitor_roast
metadata: {}
data:
cache: false
media_player_entity_id: media_player.voice_box
message: "{{visitor_roast.text}}"
target:
entity_id: tts.elevenlabs
before: sunset
after: sunrise
then:
metadata: {}
data: {}
target:
device_id: 53b3ef12acc92505d47e0f628ab40031
mode: single
I just added a similar style pipeline for the outside speaker where you get a kinder/less crass commentary and questions about why you are there, when someone is standing at the door. Together these work quite well as you get some playful banter outside, and some brutal commentary when you walk in.
This might get annoying real fast, but so far I love it. Gemini's roasts are far better than GPT4o, can't wait to try this with Grok3 in unhinged mode.
