Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

The Grymoire

Remote Input shell scripts for your Android Device, or my screen is cracked
Posted on September 17, 2014

My Android has a cracked screen. Help!

I, too, had this happen to me. I had TitaniumBackup Pro on my device, but when the new version
of KitKat came out, I lost root access because of the update. I never (smack forehead) re-rooted
my device. Therefore I didnt have everything backed up. I could use the lower third of my Nexus
7 sometimes. I tried to rotate my tablet so I could press certain spots on the uncracked screen. I
made little progress, and I am sure provided a lot of entertainment to anyone who was watching
me.

This is how I recovered my data, and what tools I wrote to make this easier. Luckily I had USB
debugging enabled.

I can remotely install applications using GooglePlay, but I needed to do more as some of these
applications require permissions set using the device. And that requires tapping on the (broken)
screen.

What didnt work

I tried to use AndroidScreenCast. This showed me the screen, but I was unable to send any input
to the device, despite the work-arounds I tried.

I also tried installing Droid VNC Server. I started the server, and I was able to click the mouse
remotely on the screen, but the VCN client on my Linux machine, Remmina Remote Desktop
Client, did not show me the screen, so I was clicking blindly.

In turned out that the best way to remotely tap on the screen was to use adb. This works very
reliably, but is inconvenient because you need to type dozens of dierent commands to do
anything useful.

Install TowelRoot, SuperSU, and Helium

Install Towelroot to root the device

I rst had to root my device. I used other methods before, but this time I used geohots TowelRoot.
I downloaded the apk, and used adb to install it:

Follow

1 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

adb install tr.apk

I had to click on the screen to permit untrusted applications to be installed. I used

adb shell input touchscreen xxx yyy

where xxx and yyy were the coordinates of the spot I wanted to press. By rotating the screen and
trying several XY combinations, I was able to approve this action. But it wasnt easy. I wanted a
better workaround. But compared to other methods for re-rooting the device, this worked

Let me briey describe my XY script, as I use it a lot. My Nexus 7 has a screen of 1200 by 1920. I
could use the touchscreen option to input, mentioned above, but I wanted an easier way to type in
locations. My home screen has a layout of 5 by 5 icons for each application, with a row on the
bottom for the apps on each screen. I wanted to be able to type XY 2 1 to click on the
application that was over 2, and down one, on this grid. My script converts this 2 and 1 location
into the command adb shell input touchscreen tap 300 350

To make my script more exible, the second version allows fractional values, so XY 2 2.5 is the
position on the second column, midway between icons 2 and 3 down (between rows 2 and 3).

Launching SuperSU

After launching SuperSU, I had to click on the OK button to grant permission:

adb shell input touchscreen tap 700 1200

# or perhaps
XY 4 4.5

To be frank, once I had SuperSU working, I was in good shape. I could remotely log
into the device, execute su, and change the permission of /data/dalvik-cache. Once
this was done, I could use adb pull /data/dalvik-cache and extract nearly everything.
But I wanted to have multiple backups.

Installing Helium Backup

I also installed Helium by ClockworkMod. I installed the Linux version on my Ubuntu system.
Once I started the application on my Linux machine, by executing ./run.sh

I launched the Helium app, and I had to grant SU access. I used the command

XY 5 4.5

Follow

2 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

using my script XY, described below, I hit the Helium Menu button, and selected the PC Download:

XY 6 0.1 # select the menu option of Helium

XY 6 0.3 # select PC Download

This launched a web server on the Android device, and gave me the URL to type into my browser.
I was then able to backup my applications using my browser on my Linux machine.

The ADB shell scripts

I wrote several independent bash shell scripts.

These are based on the adb shell input command. There are several variations to this command,
as the usage shows:

usage: input ...

input text <string>
input keyevent <key code number or name>
input [touchscreen|touchpad] tap <x> <y>
input [touchscreen|touchpad] swipe <x1> <y1> <x2> <y2>
input trackball press
input trackball roll <dx> <dy>

If one wanted to, one can type adb shell and then just type combinations of the above input
command. But thats a lot of typing. I wanted something quicker.

Here is the list of bash shell scripts in my collection below. You can combine them, extend them,
modify them, and add new commands. As an example, suppose I want to power up the device.
unlock it, go to the home screen. swipe right, launch the application at 6,2, and turn up the
volume, I can use the following bash shell script

#!/bin/bash
Power
Unlock
Home
SwipeRight
XY 6 2
sleep 5 # seconds
VolumeUp

If you know the locations of your icons, you can automate several scripts using a series of bash
scripts. Or combine scripts together to make more sophisticated scripts.

Here is a summary of the shell scripts I wrote:

Follow

3 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

Back - The Back Button

Notification - Show the list of notifications
CloseNotification - Close the notification list
GetOrientation - get the orientation of the device.
GetScreenshot - Take a screen show and stores it in screen.png
GetXY - get the X and Y dimensions of the screen
Home - The Home button
Menu - The menu button
Power - Turn power on/off
Recent - Show recent applications
SwipeDown - Swipe down
SwipeLeft - Swipe left
SwipeRight - Swipe right
SwipeUp - Swipe Up
Text - Type text - including spaces
Unlock - Unlock the device
Widgets - Show the widget
XY - tap at the X,Y location
VolumeUp - turn volume up
VolumeDown - turn volume down

Command that start with Get return values that can be used by other shell scripts.

Here is the source of each one. Each one is a small script, and some could be shell functions or
aliases, but I nd it easier to make each one a separate shell script. For conciseness, I dont show
the complete script with the bash invocation and comments. I just show the meat of the matter. As
always your mileage may vary

Back The Back Button

This is uses a lot because it undoes the last input. If you get the location wrong, this usually
returns you to the previous screen, so you can try a dierent location.

adb shell input keyevent 4

Notication Show the list of notications

adb shell input swipe 10 10 10 1000

CloseNotication Close the notication list

adb shell input swipe 10 1000 10 10

GetOrientation get the orientation of the device. Returns 0, 1, 2, or 3

Follow

4 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

adb shell dumpsys input | grep SurfaceOrientation | awk '{ print $2 }'

GetScreenshot Take a screen show and stores it in screen.png

This was taken from shevtsovs blog.

# http://blog.shvetsov.com/2013/02/grab-android-screenshot-to-computer-via.html
adb shell screencap -p | perl -pe 's/\x0D\x0A/\x0A/g' > screen.png

GetXY get the X and Y dimensions of the screen

I use this for bound checking in my XY script. It returns the maximum X and Y
positions.

adb shell dumpsys window | \

sed -n '/mUnrestrictedScreen/ s/^.*) \([0-9][0-9]*\)x\([0-9][0-9]*\)/\1 \2/p'

Home The Home button

adb shell input keyevent 3

Menu The menu button

adb shell input touchscreen tap 600 1775

Power Turn power on/o

adb shell input keyevent 26

Recent Show recent applications

adb shell input touchscreen tap 800 1850

SwipeDown Swipe down

The names of the swipe commands can be confusing. I use Left and Right to indicate that I want to
see what is on the Left or Right, not what direction my nger moves.

Follow

5 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

adb shell input swipe 800 400 1200 400

SwipeLeft Swipe left

adb shell input swipe 400 400 800 400

SwipeRight Swipe right

adb shell input swipe 800 400 400 400

SwipeUp Swipe Up

adb shell input swipe 800 400 400 400

Text Type text including spaces

Because you cannot type spaces using the adb shell test commands, I use sed to convert spaces
into %s

#!/bin/bash
str="$@"
nstr=$(echo "$str" | sed 's/ /%s/g')
echo adb shell text "$nstr"

Unlock Unlock the device

If you have a password or pin, you are on your own.

adb shell input keyevent 82

Widgets Show the widgets screen

Note this is the same event and the unlock event.

adb shell input keyevent 82

Follow

6 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

VolumeUp turn volume up

adb shell input keyevent 25

VolumeDown turn volume down

adb shell input keyevent 25

XY tap at the X,Y location

This is the most complicated routine. The rst version only accepted integer values. I added the
capability to input oating point numbers to get more precise control.

The positioning is a guestimate. I got the maximum X and Y values using GetXY. I divided these
values by the number of icons to get the XDIM and YDIM values, which represent the size of each
icon. (Improvements are welcome). I had to add an OFFSET to compensate for the rst row.

I also added a warning if the requested position is outside of the maximum dimension.

#!/bin/bash

# Bruce Barnett Wed Sep 17 13:43:51 EDT 2014

export PATH=$PATH: # make sure GetXY is in the searchpath
xmax=$(GetXY | awk '{print $1}')
ymax=$(GetXY | awk '{print $2}')

x=`echo $1 | awk '

BEGIN {
XOFF=-100
XDIM=200
}
{
newx=XOFF + ($1 * XDIM)
if (newx > XMAX) {
print "Warning: ", newx, " > ", XMAX >"/dev/stderr"
}
printf("%4.0f \n", newx )
}' XMAX=$xmax`
y=`echo $2 | awk '
BEGIN {
YOFF=100
YDIM=250
}
{
newy=YOFF + ($1 * YDIM)
if (newy > YMAX) {
print "Warning: ", newy, " > ", YMAX >"/dev/stderr"
}
printf("%4.0f\n", YOFF + ($1 * YDIM))
}' YMAX=$ymax`

Follow

7 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

echo adb shell input touchscreen tap $x $y

adb shell input touchscreen tap $x $y

Future work
If there is interest, I can make these available in a git repository.

References
http://www.tutorialsbuzz.com/2013/11/android-adb-shell-command-device.html
http://stackoverow.com/questions/15278587/pipe-into-adb-shell
http://www.netmite.com/android/mydroid/development/pdk/docs/keymaps_keyboard_input.html
dated but interesting
http://blog.shvetsov.com/2013/02/grab-android-screenshot-to-computer-via.html
http://bryanmarty.com/2013/06/09/remote-control-now-headless-android-phone/
http://developer.android.com/reference/android/view/KeyEvent.html list of key event codes

Like
One blogger likes this.

Related

The new attack vector - HID Extracting shell commands from Cataloging SDHC Cards on Ubuntu
Kali's application menu using a bash script

Gallery | This entry was posted in Hacking, Linux, Shell Scripting and tagged adb, Android, bash, Helium, Nexus 7, Titanium Backup, TowelRoot, VNC. Bookmark the permalink.

4 Responses to Remote Input shell scripts for your Android Device, or my screen is cracked

raul says:
January 8, 2015 at 4:06 pm

Very good tutorial. I wonder if there is a command to do a long press.

Follow

8 de 9 08-09-2016 09:49
Remote Input shell scripts for your Android Devic... https://grymoire.wordpress.com/2014/09/17/remo...

Reply

grymoire says:
January 9, 2015 at 2:18 am

Try the info on this link:

http://stackoverow.com/questions/11142843/how-can-i-use-adb-to-send-a-longpress-key-event
Reply

mobileqablog says:
August 27, 2016 at 8:26 pm

adb shell input keyevent 83

can be used to open and close notications.

if screen is PIN protected, you can run:

adb shell input text && adb shell input keyevent 66
however it doesnt work for all devices.
Reply

mobileqablog says:
August 27, 2016 at 8:30 pm

Recent apps: adb shell input keyevent KEYCODE_APP_SWITCH

Reply

The Grymoire
Create a free website or blog at WordPress.com.

Follow

9 de 9 08-09-2016 09:49