A blog post about how to reverse engineer a VxWorks based device.

Introduction

Embedded devices are a huge and wide world of options for CPU architectures, operating systems and file systems. You can find the combination you can think of. In this case, I'll present a study about the TP-Link TL-WR543G. This old router I had at home came equipped with a MIPS CPU and VxWorks as its OS.

In this blog post, I'll show you my adventures with the reverse engineering process of this VXWorks based device. I'm going to explain my findings and all my fails when dealing with the firmware and binary analysis, which are the main reasons I'm writing this down.

VxWorks little overview

As said previously, the WR543G router runs VxWorks. VxWorks is a proprietary RTOS developed by Wind River Systems. It supports many architectures like Intel, Power-PC, ARM, MIPS, etc. It has its own development suite called Wind River Workbench that includes an editor, compiler toolchain, debugger, and emulator.

VxWorks is used by products across a wide range of market areas: aerospace and defense, automotive, industrial such as robots, consumer electronics, medical area and networking.

As VXWorks is a proprietary OS, it has many vendor specific attributes, this includes the security model. For example, Stack Overflow Detection for user programs. The firmware structure and file system are also vendor specific.

If you are, like me, used to reverse other kinds of devices that usually include Unix-like OSes, you'll find that VxWorks requires a little more work in order to obtain the Grail.

If you want to read more about VxWorks, you can refer to the VxWorks related work section at the end of this article.

How things happened

This wasn't the first time I looked into this router. Some time ago, I tried to reverse the firmware but I didn't make much progress on that path. So, I decided to open the case and see what was inside. The hardware approach also conducted me to a dead end as I didn't want to unsolder anything at that time. So I put it aside for a while until now. I wanted to understand how the firmware was built and what could be extracted from it in order to do a deeper analysis. There isn't too much information available about VxWorks internals, just a few posts from devttys0 and some others in Chinese language. However, those were enough for me to have a very good progress with the reversing process. So, after a short introduction about the hardware analysis, I'll focus more on the firmware analysis.

Hardware analysis

Description

  • CPU: Atheros AR2317-AC1A (all integrated, WiFi, MIPS 4000)

  • Flash: Z-16PGJI-016S338-S9178PC7 (2MB?)

  • SDRAM: EM639165TS-6G, 16 MB

  • Ethernet: Marvell 88E6060-RCJ1

  • UART: Yes

Regarding the flash memory, the previous version (WR542G) was 2 MB in size. I couldn't find too much info with the serial number so I suppose the size of the flash is similar in this case.

The following are some pictures from the main board.

Front

Main board: front

Back

Main board: back

CPU, RAM, Flash

Main board: chips close-up

Serial port

When looking at the main board, you can see that there's a pinout similar to UART (Universal asynchronous receiver-transmitter). I followed the methodology mentioned in this devttys0's blog post in order to reverse engineer the pins. You can see them in the following picture:

UART port on PCB

These are the UART parameters: COM3 (in my case, check yours), Baudrate: 38400

Putty configuration

Once I set the UART interface up, I was able to access the boot log and boot menu:

AR2315 rev 0x00000090 startup...
Attached TCP/IP interface to ae unit 0
Attaching interface lo0...done

USRCONF : g_size = 18112
Name = MODULE_USR_CONF_T , size = 12
Name = UC_IEEE802_1X_CFG_DATA_T , size = 512
Name = UC_ADVANCED_CFG_T , size = 16
Name = UC_ARP_CFG_T , size = 32
Name = UC_BPA_CFG_DATA_T , size = 912
Name = UC_DDNS_T , size = 1480
Name = UC_DHCPC_CFG_DATA_T , size = 416
Name = UC_DHCPS_CFG_AND_STATIC_T , size = 708
Name = UC_FIRE_WALL_STATE_T , size = 1992
Name = UC_FORWARD_VIRTUAL_SERVE_CLASS_T , size = 1572
Name = UC_IFQOS_ROUTER_CONF_FLASH_T , size = 120
Name = UC_LAN_CFG_DATA_T , size = 8
Name = UC_L2TP_CFG_DATA_T , size = 1472
Name = UC_MAC_CONFIG_T , size = 80
Name = UC_PPPOE_CFG_DATA_T , size = 1568
Name = UC_NTP_PREFER_SRV_CFG_DATA_T , size = 28
Name = UC_STATIC_IP_CFG_DATA_T , size = 144
Name = UC_SATTIC_ROUTE_CFG_DATA_T , size = 328
Name = UC_MANAGE_USERS_T , size = 64
Name = UC_UTILITIES_T , size = 16
Name = UC_WANCONNTYPE_T , size = 16
Name = UC_WLAN_CFG_T , size = 5020
Name = UC_PPTP_CFG_DATA_T , size = 1472
Name = UC_NETWORK_PSEUDO_T , size = 4
Name = UC_IFQOS_ROUTER_CONF_FLASH_T , size = 120
Port QoS..................
qos_IFInit..................
qos_IFLoadFromFlash..................
wireless access point starting...
wlan0 Ready


entering tddp...



                                                        Software Platform for ARM
  Copyright(C) 2001-2004 by TP-LINK TECHNOLOGIES CO., LTD.
  Creation date: Sep  4 2008, 17:18:51

  Press CTRL-B to enter bootmenu...

  Boot Menu:
         1:  Download application program
         2:  Modify Bootrom password
         3:  Exit the menu
         4:  Reboot
         5:  User commond line

One interesting thing is that the boot log refers to AR2315 instead of AR2317. The AR2315 is a MIPS 4KE 32-bit with 183 MHz processor.

Another important thing to mention is that the boot menu doesn't work at all. If you press any of the reported commands, nothing happens (Btw, I'm using an English keyboard, if you ask). However, when trying other keys, we obtain the following results:

  • n: netstat

  Boot Menu:
         1:  Download application program
         2:  Modify Bootrom password
         3:  Exit the menu
         4:  Reboot
         5:  User commond line
                Enter your choice(1-4):n
413
Active Internet connections (including servers)
PCB      Proto Recv-Q Send-Q  Local Address      Foreign Address    (state)
-------- ----- ------ ------  ------------------ ------------------ -------
80fb7f08 TCP        0      0  0.0.0.0.80            0.0.0.0.0             LISTEN
80fb8538 UDP        0      0  0.0.0.0.2050          0.0.0.0.0
80fb84b4 UDP        0      0  0.0.0.0.53            0.0.0.0.0
80fb8430 UDP        0      0  0.0.0.0.67            0.0.0.0.0
80fb83ac UDP        0      0  0.0.0.0.68            0.0.0.0.0

It seems an HTTP server is running. This is the report from nmap:

Nmap scan report for 192.168.1.1

Host is up (0.0044s latency).

Not shown: 98 filtered ports

PORT     STATE  SERVICE

80/tcp   open   http

1900/tcp closed upnp

MAC Address: 00:27:19:C1:76:52 (Tp-link Technologies)
  • i: ifconfig

  Boot Menu:
         1:  Download application program
         2:  Modify Bootrom password
         3:  Exit the menu
         4:  Reboot
         5:  User commond line
                Enter your choice(1-4):i

ae (unit number 0):
         Flags: (0x8b63) UP BROADCAST MULTICAST PROMISCUOUS ARP RUNNING
         Type: ETHERNET_CSMACD
         Internet address: 192.168.1.1
         Broadcast address: 192.168.1.255
         Netmask 0xffffff00 Subnetmask 0xffffff00
         Ethernet address is 00:27:19:c1:76:52
         Metric is 0
         Maximum Transfer Unit size is 1500
         0 octets received
         126 octets sent
         0 packets received
         3 packets sent
         0 non-unicast packets received
         0 non-unicast packets sent
         0 unicast packets received
         3 unicast packets sent
         0 input discards
         0 input unknown protocols
         0 input errors
         0 output errors
         0 collisions; 0 dropped
lo (unit number 0):
         Flags: (0x8069) UP LOOPBACK MULTICAST ARP RUNNING
         Type: SOFTWARE_LOOPBACK
         Internet address: 127.0.0.1
         Netmask 0xff000000 Subnetmask 0xff000000
         Metric is 0
         Maximum Transfer Unit size is 32768
         1 packets received; 1 packets sent
         0 multicast packets received
         0 multicast packets sent
         0 input errors; 0 output errors
         0 collisions; 0 dropped
ae (unit number 1):
         Flags: (0x8b63) UP BROADCAST MULTICAST PROMISCUOUS ARP RUNNING
         Type: ETHERNET_CSMACD
         Internet address: 10.1.177.242
         Broadcast address: 10.1.177.243
         Netmask 0xff000000 Subnetmask 0xfffffffc
         Ethernet address is 00:1b:fc:0e:b1:f2
         Metric is 0
         Maximum Transfer Unit size is 1500
         13696 octets received
         27868 octets sent
         31 packets received
         63 packets sent
         0 non-unicast packets received
         31 non-unicast packets sent
         31 unicast packets received
         32 unicast packets sent
         0 input discards
         0 input unknown protocols
         0 input errors
         0 output errors
         0 collisions; 0 dropped
ppp (unit number 1):
         Flags: (0xb0) DOWN POINT-TO-POINT
         Type: PPP
         Metric is 0
         Maximum Transfer Unit size is 1400
         0 octets received
         0 octets sent
         0 packets received
         0 packets sent
         0 non-unicast packets received
         0 non-unicast packets sent
         0 unicast packets received
         0 unicast packets sent
         0 input discards
         0 input unknown protocols
         0 input errors
         0 output errors
  • m: meminfo

  Boot Menu:
         1:  Download application program
         2:  Modify Bootrom password
         3:  Exit the menu
         4:  Reboot
         5:  User commond line
                Enter your choice(1-4):m
501

FREE LIST:
   num    addr       size
  ---- ---------- ----------
         1 0x80b693d0         64
         2 0x80ffb1f0      13056
         3 0x80ff6770         96
         4 0x80b66b60        976
         5 0x8038df30    8062064
         6 0x80f9fb80        240


SUMMARY:
 status    bytes     blocks   avg block  max block
 ------ ---------- --------- ---------- ----------
current
   free    8076496         6    1346082    8062064
  alloc    4973520      1459       3408          -
cumulative
  alloc    4992304      1523       3277          -
  • k: process list

  Boot Menu:
         1:  Download application program
         2:  Modify Bootrom password
         3:  Exit the menu
         4:  Reboot
         5:  User commond line
                Enter your choice(1-4):k


  NAME        ENTRY       TID    PRI   STATUS      PC       SP     ERRNO  DELAY
---------- ------------ -------- --- ---------- -------- -------- ------- -----
 --tExcTask   801c19c4     80ff5380   0 PEND       801e3c50 80ff5260       0     0
-tLogTask   801c6c54     80ff27f0   0 PEND       801e3c50 80ff26d8     --- ---------- --------- --------- ----------
current
   free   8076496         6    346082    8062064  alloc    4973520      1459       3408          -
cumulative
  alloc    4992304      1523     377          -
  0     0
bootromTask8005ad30     80b65be0   8 READY      801dc998 80b64e40  3d0002     0
tWlanCal   800ef730     80be2ba0  10 PEND       801e3c50 80be2ab0       0     0
tApCserv   800de55c     80b6c8f0  10 PEND       801e3c50 80b6c800       0     0
Detectd    80022638     80b631d0  40 DELAY      801db708 80b63148       0    48
timerManage8006d7f0     80d44780  48 PEND+T     80158414 80d44708  3d0004     2
endRecvTask8005bce4     80b85020  49 PEND+T     801e3c50 80b84f10  3d0004     1
tNetTask   8019aab4     80fa4c90  50 PEND       80158414 80fa4c00       0     0
tApHouseKee800de9d0     80b74db0  50 DELAY      801db708 80b74d00       0  1533
dot1xTask  800f3a3c     80b6fd20  50 PEND+T     80158414 80b6fa90  3d0002     6
dhcpcState 800b4438     80de18d0  56 PEND       80158414 80de1830       0     0
dhcpcRead  800b4e10     80de02d0  56 PEND       80158414 80de00b8  3d0002     0
swDhcpcd   80062620     80dde870  56 PEND       801e3c50 80dde618       0     0
802_1X     80093c40     80b680e0 150 PEND+T     80158414 80b68060  3d0004    26
swL2tpd    80069944     80df8d80 198 PEND+T     801e3c50 80df8b08  3d0004    51
swPptpd    8006b4c4     80df2130 198 PEND+T     801e3c50 80df1eb8  3d0004    50
swPppoed   8006863c     80dfd670 199 PEND+T     801e3c50 80dfd3b0  3d0004    48
pppoed_0   8006fbd8     80dff8e0 200 PEND+T     801e3c50 80dff7c8  3d0004    17
pptpProc_0 8007aeac     80df43a0 200 PEND+T     801e3c50 80df4070  3d0004     1
l2tpd_0    80075bd4     80dfaff0 201 PEND+T     801e3c50 80dfaed8  3d0004    15
pptpd_0    800785f0     80df6610 201 PEND+T     801e3c50 80df64f0  3d0004    13
usrRebootd 80066648     80d452b0 202 PEND+T     801e3c50 80d451d0  3d0004    42
mud0_80    8002dd40     80f0ebf0 203 PEND+T     80158414 80f0e9b0  3d0002   581
tFWCONF    8008fbd8     80ddc960 203 DELAY      801db708 80ddc8d8       0   279
dnsProxyd  800c0728     80d475e0 204 PEND+T     80158414 80d473e0  3d0002    38
sntpd      800bf518     80d48e80 205 PEND+T     801e3c50 80d48d48  3d0004    37
sysExLogd  800201b0     80f388a0 240 PEND       80158414 80f38488  3d0002     0
dynTask    800c35fc     80d424d0 250 DELAY      801db708 80d42428       0    35
BPAd_0     800cb6e0     80decc80 253 DELAY      801db708 80decb48       0    33
dhcpsd     800a7688     80d4b3a0 254 PEND+T     80158414 80d4b178  3d0002    32
BPAm_0     800ce5c0     80dea210 255 DELAY      801db708 80dea170       0    31

However, I didn't find a way to get a shell through this menu :(

Firmware analysis

Downloads:

Binwalk analysis

My first try with Binwalk gave me these results:

fastix@bulin:~/wr543g$ binwalk wr543gv2-en-up.bin

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
20132         0x4EA4          Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
20228         0x4F04          VxWorks operating system version "5.5.1" , compiled: "Sep  4 2008, 17:18:57"
20324         0x4F64          Copyright string: "Copyright 1984-2002 Wind River Systems, Inc."
21685         0x54B5          Zlib compressed data, default compression

As seen in the output from Binwalk, there is a compilation header showing a string related to VxWorks, specifically, it seems that we are in front of VxWorks 5.5.1.

There is also a Zlib compressed stream at the end, starting at offset 0x54B5. There are no signs of bootloader, filesystem, kernel, etc; probably, and with some luck, something is going to appear in the compressed stream.

In order to extract the Zlib compressed stream I just used the command binwalk -e file.bin.

By looking at the strings on the uncompressed stream, there are some interesting details:

ae(0,0)TP-MIPS:vxWorks h=192.168.1.18 e=192.168.1.5:0xffffff00 u=wr541 pw=123 f=0x0 tn=wr541 o=ae s=factory
resetting to factory config.

Apparently, when factory resetting the router, the default username and password combination is wr541/123.

If we use Binwalk again over the uncompressed stream, here is the result:

fastix@bulin:~/wr543g/_wr543gv2-en-up.bin.extracted$ binwalk 54B5

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
880412        0xD6F1C         Certificate in DER format (x509 v3), header length: 4, sequence length: 1
880432        0xD6F30         Certificate in DER format (x509 v3), header length: 4, sequence length: 4
1916528       0x1D3E70        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
1917168       0x1D40F0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
1917808       0x1D4370        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
1918648       0x1D46B8        Unix path: /Tornado2.2.1-mips/target/config/ar2315/utility.c, line 393
1918760       0x1D4728        Unix path: /Tornado2.2.1-mips/target/config/ar2315/utility.c, line 408
1918852       0x1D4784        Unix path: /Tornado2.2.1-mips/target/config/ar2315/utility.c, line 423
1918944       0x1D47E0        Unix path: /Tornado2.2.1-mips/target/config/ar2315/utility.c, line 468
1919018       0x1D482A        Unix path: /depot/sw/branches/mBSSID_dev/src/ap/os/vxworks/target/config/ar531xPlus/ae531xEnd.c#1 $
1919594       0x1D4A6A        Unix path: /Tornado2.2.1-mips/target/config/ar2315/ae531xEnd.c, line 319
1919744       0x1D4B00        Unix path: /Tornado2.2.1-mips/target/config/ar2315/ae531xEnd.c, line 703
1920008       0x1D4C08        Unix path: /Tornado2.2.1-mips/target/config/ar2315/ae531xEnd.c, line 2188
1922144       0x1D5460        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
1923290       0x1D58DA        Unix path: /depot/sw/branches/mBSSID_dev/src/ap/os/vxworks/target/config/ar531xPlus/sysLib.c#1 $
1937472       0x1D9040        CRC32 polynomial table, big endian
1938879       0x1D95BF        Copyright string: "Copyright 1995-2002 Jean-loup Gailly "
1939471       0x1D980F        Copyright string: "Copyright 1995-2002 Mark Adler "
1945225       0x1DAE89        HTML document footer
1945892       0x1DB124        HTML document header
1947067       0x1DB5BB        HTML document footer
1947076       0x1DB5C4        HTML document header
1947285       0x1DB695        HTML document footer
1947352       0x1DB6D8        HTML document header
1947392       0x1DB700        HTML document header
1949774       0x1DC04E        HTML document footer
1954280       0x1DD1E8        Unix path: /rc_filesys/doc/dynaform/common.js
1954508       0x1DD2CC        Unix path: /rc_filesys/doc/frames/top.htm
1959104       0x1DE4C0        HTML document header
1960860       0x1DEB9C        HTML document footer
1960868       0x1DEBA4        GIF image data, version "87a", 16 x 41
1968256       0x1E0880        GIF image data, version "89a", 220 x 146
1973772       0x1E1E0C        GIF image data, version "89a", 10 x 10
1973832       0x1E1E48        GIF image data, version "89a", 10 x 10
1973896       0x1E1E88        GIF image data, version "89a", 160 x 72
1978532       0x1E30A4        GIF image data, version "89a", 10 x 10
1978592       0x1E30E0        JPEG image data, JFIF standard 1.01
1982668       0x1E40CC        JPEG image data, JFIF standard 1.01
1992460       0x1E670C        JPEG image data, JFIF standard 1.02
1992490       0x1E672A        TIFF image data, little-endian offset of first image directory: 8
1993684       0x1E6BD4        JPEG image data, JFIF standard 1.01
1994112       0x1E6D80        HTML document header
1994236       0x1E6DFC        HTML document footer
1997632       0x1E7B40        HTML document header
2006161       0x1E9C91        HTML document footer
2010912       0x1EAF20        HTML document header
2013167       0x1EB7EF        HTML document footer
2018806       0x1ECDF6        Copyright string: "Copyright(C) 2001-2004 by %s"
2023168       0x1EDF00        Unix path: /core/bsp/archives/ar2315/ar531xPlusreg.h-arc   1.5   01 Jun 2006 11:40:30   dai  $
2032413       0x1F031D        Copyright string: "Copyright 2000 Wind River Systems, Inc."
2032645       0x1F0405        Copyright string: "Copyright 2001 Wind River Systems, Inc."
2032737       0x1F0461        Copyright string: "Copyright 2001 Wind River Systems, Inc."
2043168       0x1F2D20        XML document, version: "1.0"
2043572       0x1F2EB4        XML document, version: "1.0"
2047104       0x1F3C80        HTML document header
2047136       0x1F3CA0        HTML document footer
2061824       0x1F7600        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2069248       0x1F9300        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2069760       0x1F9500        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2070160       0x1F9690        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2072000       0x1F9DC0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2072688       0x1FA070        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2074288       0x1FA6B0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2074784       0x1FA8A0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2075088       0x1FA9D0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2075584       0x1FABC0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2076944       0x1FB110        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2078192       0x1FB5F0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2080528       0x1FBF10        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2082896       0x1FC850        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2085056       0x1FD0C0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2086560       0x1FD6A0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2087408       0x1FD9F0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2087936       0x1FDC00        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2089984       0x1FE400        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2090224       0x1FE4F0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2090464       0x1FE5E0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2090816       0x1FE740        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2091328       0x1FE940        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2091558       0x1FEA26        Unix path: /depot/sw/releases/4.1.2/src/hal/halUtil.c#1 $
2092010       0x1FEBEA        Unix path: /depot/sw/releases/4.1.2/src/hal/halBeacon.c#1 $
2095206       0x1FF866        Unix path: /depot/sw/releases/4.1.2/src/hal/ar5212/ar5212Attach.c#9 $
2099696       0x2009F0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2100512       0x200D20        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2101040       0x200F30        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2101600       0x201160        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2103120       0x201750        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2103376       0x201850        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2103888       0x201A50        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2105808       0x2021D0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2106592       0x2024E0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2107296       0x2027A0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2107920       0x202A10        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2118544       0x205390        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2121168       0x205DD0        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2121488       0x205F10        Unix path: /core/bsp/archives/ar2315/ar531xPlus.h-arc   1.4   Dec 27 2005 15:00:04   dai  $
2159077       0x20F1E5        Unix path: /../ip_qos/src/config/qosTask.c, line 102
2159268       0x20F2A4        Unix path: /../ip_qos/src/config/qosTask.c, line 252
2159348       0x20F2F4        Unix path: /../ip_qos/src/config/qosTask.c, line 275
2159446       0x20F356        Unix path: /../ip_qos/src/config/qosTask.c, line 303
2159540       0x20F3B4        Unix path: /../ip_qos/src/config/qosTask.c, line 315
2159633       0x20F411        Unix path: /../ip_qos/src/config/qosTask.c, line 323
2159736       0x20F478        Unix path: /../ip_qos/src/config/qosTask.c, line 370
2159816       0x20F4C8        Unix path: /../ip_qos/src/config/qosTask.c, line 417
2159903       0x20F51F        Unix path: /../ip_qos/src/config/qosTask.c, line 443
2160026       0x20F59A        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 206
2160114       0x20F5F2        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 217
2160198       0x20F646        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 254
2160295       0x20F6A7        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 364
2160368       0x20F6F0        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 460
2160471       0x20F757        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 488
2160564       0x20F7B4        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 491
2160672       0x20F820        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 501
2160774       0x20F886        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 502
2160848       0x20F8D0        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 583
2160953       0x20F939        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 776
2161057       0x20F9A1        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 813
2161159       0x20FA07        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 830
2161261       0x20FA6D        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 850
2161365       0x20FAD5        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 886
2161465       0x20FB39        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 1088
2161553       0x20FB91        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 1253
2161650       0x20FBF2        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 1256
2161731       0x20FC43        Unix path: /../ip_qos/src/core/qos_coreSch.c, line 1274
2161873       0x20FCD1        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 97
2161957       0x20FD25        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 141
2162043       0x20FD7B        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 147
2162146       0x20FDE2        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 164
2162233       0x20FE39        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 278
2162317       0x20FE8D        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 285
2162404       0x20FEE4        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 297
2162492       0x20FF3C        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 298
2162606       0x20FFAE        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 394
2162693       0x210005        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 483
2162777       0x210059        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 490
2162864       0x2100B0        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 506
2162952       0x210108        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 507
2163042       0x210162        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 516
2163135       0x2101BF        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 519
2163226       0x21021A        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 520
2163318       0x210276        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 523
2163405       0x2102CD        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 534
2163489       0x210321        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 536
2163719       0x210407        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 640
2163815       0x210467        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 721
2163896       0x2104B8        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 747
2163984       0x210510        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 782
2164066       0x210562        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 835
2164266       0x21062A        Unix path: /../ip_qos/src/if_config/qos_IFConfig.c, line 880
2164484       0x210704        Unix path: /../ip_qos/src/if_config/testcase.c, line 180
2166310       0x210E26        Unix path: /core/bsp/archives/ar2315/ae531xEnd.h-arc   1.4   10 Dec 2005 10:03:34   lqm  $
2166996       0x2110D4        Unix path: /../ip_qos/src/core/qos_coreAckFirstFifo.c, line 194
2167160       0x211178        Unix path: /../ip_qos/src/core/qos_coreAckFirstFifo.c, line 229
2167274       0x2111EA        Unix path: /../ip_qos/src/common/memPool.c, line 32
2167356       0x21123C        Unix path: /../ip_qos/src/common/memPool.c, line 40
2167440       0x211290        Unix path: /../ip_qos/src/common/memPool.c, line 63
2167605       0x211335        Unix path: /../ip_qos/src/common/memPool.c, line 67
2167701       0x211395        Unix path: /../ip_qos/src/common/memPool.c, line 68
2167784       0x2113E8        Unix path: /../ip_qos/src/common/memPool.c, line 74
2167868       0x21143C        Unix path: /../ip_qos/src/common/memPool.c, line 80
2167954       0x211492        Unix path: /../ip_qos/src/common/memPool.c, line 86
2168038       0x2114E6        Unix path: /../ip_qos/src/common/memPool.c, line 92
2168135       0x211547        Unix path: /../ip_qos/src/common/memPool.c, line 120
2168234       0x2115AA        Unix path: /../ip_qos/src/common/memPool.c, line 139
2168331       0x21160B        Unix path: /../ip_qos/src/config/qos_Config.c, line 59
2168438       0x211676        Unix path: /../ip_qos/src/config/qos_Config.c, line 68
2168542       0x2116DE        Unix path: /../ip_qos/src/config/qos_Config.c, line 89
2168630       0x211736        Unix path: /../ip_qos/src/config/qos_Config.c, line 136
2168716       0x21178C        Unix path: /../ip_qos/src/config/qos_Config.c, line 289
2168808       0x2117E8        Unix path: /../ip_qos/src/config/qos_Config.c, line 366
2168896       0x211840        Unix path: /../ip_qos/src/config/qos_Config.c, line 394
2168984       0x211898        Unix path: /../ip_qos/src/config/qos_Config.c, line 418
2169173       0x211955        Unix path: /../ip_qos/src/config/qos_htbConfig.c, line 262
2169296       0x2119D0        Unix path: /../ip_qos/src/config/qos_htbConfig.c, line 328
2169705       0x211B69        Unix path: /../ip_qos/src/config/qos_htbDump.c, line 184
2169805       0x211BCD        Unix path: /../ip_qos/src/config/qos_htbDump.c, line 187
2169957       0x211C65        Unix path: /../ip_qos/src/config/qos_htbDump.c, line 208
2170171       0x211D3B        Unix path: /../ip_qos/src/config/qos_htbDump.c, line 221
2170260       0x211D94        Unix path: /../ip_qos/src/core/qos_coreFifo.c, line 62
2170344       0x211DE8        Unix path: /../ip_qos/src/core/qos_coreFifo.c, line 87
2170428       0x211E3C        Unix path: /../ip_qos/src/core/qos_coreFifo.c, line 118
2170512       0x211E90        Unix path: /../ip_qos/src/core/qos_coreFifo.c, line 169
2170644       0x211F14        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 117
2170732       0x211F6C        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 146
2170859       0x211FEB        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 192
2170972       0x21205C        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 199
2171071       0x2120BF        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 205
2171173       0x212125        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 206
2171271       0x212187        Unix path: /../ip_qos/src/core/qos_coreGeneric.c, line 213
2171371       0x2121EB        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 217
2171459       0x212243        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 237
2171533       0x21228D        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 1414
2171648       0x212300        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 1669
2171737       0x212359        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 1683
2171958       0x212436        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 2527
2172160       0x212500        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 2728
2172263       0x212567        Unix path: /../ip_qos/src/core/qos_coreHtb.c, line 2732
2172374       0x2125D6        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 33
2172467       0x212633        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 34
2172555       0x21268B        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 48
2172652       0x2126EC        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 49
2172739       0x212743        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 66
2172852       0x2127B4        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 67
2172939       0x21280B        Unix path: /../ip_qos/src/if_config/qos_IFClassifier.c, line 138
2173063       0x212887        Unix path: /../ip_qos/src/common/qos_map.c, line 24
2173147       0x2128DB        Unix path: /../ip_qos/src/common/qos_map.c, line 27
2173231       0x21292F        Unix path: /../ip_qos/src/common/qos_map.c, line 43
2173313       0x212981        Unix path: /../ip_qos/src/common/qos_map.c, line 53
2173398       0x2129D6        Unix path: /../ip_qos/src/common/qos_map.c, line 80
2173480       0x212A28        Unix path: /../ip_qos/src/common/qos_map.c, line 82
2173572       0x212A84        Unix path: /../ip_qos/src/common/qos_map.c, line 98
2174160       0x212CD0        VxWorks operating system version "5.5.1" , compiled: "Sep  4 2008, 17:18:51"
2191631       0x21710F        HTML document footer
[..]
[cut for brevity. What followed here were a lot of HTML document footers]
[..]
2556403       0x2701F3        Copyright string: "Copyright 1999, Mark Martinec. Frontier Artistic License applies."
2558288       0x270950        XML document, version: "1.0"
2561036       0x27140C        XML document, version: "1.0"
2561868       0x27174C        XML document, version: "1.0"
2563832       0x271EF8        XML document, version: "1.0"
2575992       0x274E78        Base64 standard index table
2576304       0x274FB0        XML document, version: "1.0"
2580992       0x276200        Base64 standard index table
2661593       0x289CD9        Copyright string: "Copyright 1984-2002 Wind River Systems, Inc."
2666394       0x28AF9A        VxWorks WIND kernel version "2.6"

OK, a lot of HTMLs/GIFs/XMLs/etc, a lot of Web stuff. Even, if we grep a little bit more, we find references to HTML files required to call each of the functions available from the Web GUI:

/userRpm/FirmwareUpdateTemp.htm
/userRpm/ConfUpdateTemp.htm
/userRpm/MenuRpm.htm
/userRpm/MainRpm.htm
/userRpm/StatusRpm.htm
/userRpm/StatusHelpRpm.htm
/userRpm/WzdStartRpm.htm
/userRpm/WzdOpModeRpm.htm
/userRpm/WzdWanTypeRpm.htm
/userRpm/WzdStaticIpRpm.htm
/userRpm/WzdDynaIpRpm.htm
/userRpm/WzdPPPoERpm.htm
/userRpm/WzdEndRpm.htm
/userRpm/WzdStaticIpHelpRpm.htm
/userRpm/WzdStartHelpRpm.htm
/userRpm/WzdOpModeHelpRpm.htm
/userRpm/WzdWanTypeHelpRpm.htm
/userRpm/WzdPPPoEHelpRpm.htm
/userRpm/WzdFinishHelpRpm.htm
/userRpm/WzdWlanRpm.htm
/userRpm/WzdWlanHelpRpm.htm
/userRpm/NatDebugRpm26525557.htm
/userRpm/NetworkCfgRpm.htm
/userRpm/WanStaticIpCfgRpm.htm
/userRpm/WanDynamicIpCfgRpm.htm
/userRpm/WanCfgRpm.htm
/userRpm/PPPoECfgRpm.htm
/userRpm/PPPoECfgAdvRpm.htm
/userRpm/MacCloneCfgRpm.htm
/userRpm/WanBpaCfgRpm.htm
/userRpm/WanBpaCfgHelpRpm.htm
/userRpm/L2TPCfgRpm.htm
/userRpm/L2tpCfgHelpRpm.htm
/userRpm/PPTPCfgRpm.htm
/userRpm/PptpCfgHelpRpm.htm
/userRpm/NetworkCfgHelpRpm.htm
/userRpm/WanStaticIpCfgHelpRpm.htm
/userRpm/WanDynamicIpCfgHelpRpm.htm
/userRpm/PPPoECfgHelpRpm.htm
/userRpm/MacCloneCfgHelpRpm.htm
/userRpm/WanStaticIpCfgRpm_8021X.htm
/userRpm/WanDynamicIpCfgRpm_8021X.htm
...

Also, by looking at the strings, we can derive interesting information related to logins:

Strings present in the firmware

The Hello123World string, apparently, is the password for the PPPoE service login.

All the things we found are good things so far, but where are the binaries? There are no signs of code nor other compressed streams that can contain them. At this point, I was thinking that if this Zlib uncompressed stream only contains Web stuff, how are binary files updated in the router? It happens that, sometimes, IoT vendors publish two types of firmwares for their devices, one that holds the Web stuff and another firmware file that contains the binary files. This is not the case but it happens, for example, with IP cameras.

In order to make sure I wasn't missing something, I checked the old firmware files I had for this device but the results where the same, same structure, only one Zlib compressed stream containing Web stuff. At this point I was thinking that, maybe, as I was dealing with a proprietary OS, the firmware file does not include some parts, probably, the boot loader is missing from the firmware update file. However, other stuff as the file system must be present thus some binary code should be also present too. Are these binaries compressed, encrypted or obfuscated in some way?

I just did a quick check using Binwalk again to see if I could find any code in the uncompressed stream. Binwalk has the -Y switch:

Attempts to identify the CPU architecture of executable code contained in a file using the capstone disassembler

This was the result:

fastix@bulin:~/wr543g/_wr543gv2-en-up.bin.extracted$ binwalk -Y 54B5

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             MIPS executable code, 32/64-bit, big endian, at least 1250 valid instructions

Bingo! some MIPS code, big endian, is present starting at offset 0.

OK, so far, we have some Web stuff mixed with some MIPS code. This is not weird, embedded devices commonly implement a single binary that acts as different servers (HTTP, FTP, etc;) and has embedded data, as in this case.

Then, I tried to load the uncompressed stream in IDA, I selected the MIPS big endian [mipsb] processor type and immediately, IDA showed a new window requesting some data from me to load the file:

IDA Disassembly memory organization

As I didn't have any, I accepted the default values and another warning was shown:

IDA warning about missing entry point

Basically, our entry point is missing and IDA don't know where to start loading the file. OK, no problem. Let's try to disassemble the first bytes:

IDA disassembly attempt

Great, IDA was able to identify one function and some code is shown in our disassembly window. However, it seems that it will be a hard work to reverse engineer the file with just that :P. Apparently, we have to dig deeper and find the entry point to load the file.

It's interesting to note one thing: the address 0x802A3960 is moved into the $gp register. This register is a global pointer that points into the middle of a 64K block of memory in the heap that holds constants and global variables. The mentioned address gave me an idea about the memory range I could look for.

So, I returned to the firmware analysis again, trying to find something that can help me to progress a little bit more, and I found two more interesting strings:

IDA Disassembly memory organization, second attemptIDA disassembly second attempt

Both strings are "OWOW.." but with different lengths. The first occurence is 16 bytes long while the second one is 32 bytes long. My experience with file formats analysis was telling me that these strings could be a kind of magic bytes (bytes at some point in a file, mostly at the beginning of the file, that are used to identify the file format i.e: PE, GIF, PNG, etc). I googled a little bit and found a very good reference in the devttys0's blog that helped me a lot with the analysis. In the same blog post, there is another reference to a post from Ruben Santamarta from IOActive with some more details about these signature bytes.

The VxWorks MemFS file system

According to the previously mentioned posts, the OWOW.. string is used as a signature in the MemFS Wind River management file system. The string may be found twice in the firmware, according to IOactive's blog spot. I'll quote the paragraph:

the first one due to  the .h  where it is defined (close to other strings such as the webserver banner )and the second one already as part of  the MemFS.

In the first place, devttys0 presented the following stucture for the header:

struct owfs_header
{
   char magic[32];           // 'OWOWOWOWOWOW...'
   uint32_t version;         // version #1
   uint32_t file_count;
   uint32_t unknown;         // ??
}

OWFS is how devttys0 named it on the post at first when nobody knew this file system was called MemFS.

In my case, this definition didn't match with what I had:

Filesystem hex dump

As you can see, I had only two DWORDS before the file entries. I went back to the other post and found that the third field can be optional. Then, the header in my case looked more like this:

struct owfs_header
{
   char magic[32];           // 'OWOWOWOWOWOW...'
   uint32_t version;         // version #1
   uint32_t file_count;      // 16 files
   array filenames_plus_eof[file_count];
}

The flag field mentioned in the IOActive's blog post doesn't make sense for my firmware file. The file list after the firmware signature contains 0x10 entries. Each entry has a maximum of 0x4C bytes in which 0x40 bytes are reserved for the filenames, 4 bytes for the file size, 4 bytes for the file offset (from the beginning of the file system header) and 4 bytes for an unknown field (null in all cases). The interesting thing is that all the files in the list are in plain, meaning that the flag field (1 in my case) doesn't represent the compressed/uncompressed option. It seems more like a version field or the values are used in the opposite way meaning 1 = plain, 2 = compressed.

In my case, the file entry header looks like this:

struct FILE_ENTRY_HEADER
{
        char filename[0x40];
        uint32_t file_size;
        uint32_t file_offset;
        uint32_t reserved; // always 0
}

It's almost the same definition as in the devttys0's post except for the last field always equal to null. It seems that the size of the header varies depending on the device/firmware.

With all this information, I was able to write a Python script to extract the files from the MemFS file system:

C:\Users\itsevart\Documents\reversing-tp-link-wr543g\code>python extract_files.py ..\bin\54B5
[+] Extracting files from ..\bin\54B5
[+] Searching signature..
[+] Signature found at offset: 1dd1c0
[+] SIGNATURE: OWOWOWOWOWOWOWOWOWOWOWOWOWOWOWOW
[+] Flag: 0x00000001
[+] File count: 16
[+] Reading file entries..
>> Entry no 0
[+] filename: /rc_filesys/doc/dynaform/common.js
[+] file size: 915 bytes
[+] file offset: 0x000004e8
>> Entry no 1
[+] filename: /rc_filesys/doc/dynaform/css_help.css
[+] file size: 627 bytes
[+] file offset: 0x0000087c
>> Entry no 2
[+] filename: /rc_filesys/doc/dynaform/css_main.css
[+] file size: 2063 bytes
[+] file offset: 0x00000af0
>> Entry no 3
[+] filename: /rc_filesys/doc/frames/top.htm
[+] file size: 1763 bytes
[+] file offset: 0x00001300
>> Entry no 4
[+] filename: /rc_filesys/doc/images/blue.gif
[+] file size: 62 bytes
[+] file offset: 0x000019e4
>> Entry no 5
[+] filename: /rc_filesys/doc/dynaform/commonfuncsEn1.js
[+] file size: 3675 bytes
[+] file offset: 0x00001a24
>> Entry no 6
[+] filename: /rc_filesys/doc/dynaform/commonfuncsEn3.js
[+] file size: 3648 bytes
[+] file offset: 0x00002880
>> Entry no 7
[+] filename: /rc_filesys/doc/images/helpPic.gif
[+] file size: 5515 bytes
[+] file offset: 0x000036c0
>> Entry no 8
[+] filename: /rc_filesys/doc/images/minus.gif
[+] file size: 59 bytes
[+] file offset: 0x00004c4c
>> Entry no 9
[+] filename: /rc_filesys/doc/images/plus.gif
[+] file size: 63 bytes
[+] file offset: 0x00004c88
>> Entry no 10
[+] filename: /rc_filesys/doc/images/productphoto.gif
[+] file size: 4634 bytes
[+] file offset: 0x00004cc8
>> Entry no 11
[+] filename: /rc_filesys/doc/images/pw.gif
[+] file size: 60 bytes
[+] file offset: 0x00005ee4
>> Entry no 12
[+] filename: /rc_filesys/doc/images/top1_1.jpg
[+] file size: 4076 bytes
[+] file offset: 0x00005f20
>> Entry no 13
[+] filename: /rc_filesys/doc/images/top1_2.jpg
[+] file size: 9790 bytes
[+] file offset: 0x00006f0c
>> Entry no 14
[+] filename: /rc_filesys/doc/images/top2.jpg
[+] file size: 1221 bytes
[+] file offset: 0x0000954c
>> Entry no 15
[+] filename: /rc_filesys/doc/images/top_bg.jpg
[+] file size: 289 bytes
[+] file offset: 0x00009a14

Even though I was able to extract several files, 16 in total, I thought there were very few. In fact, remember that I showed a lot more references to .htm files. I started to search a little bit more and noted that there were lots of files scattered in the firmware data that didn't follow any structure and didn't contain any specific header so far. Some of these files contained a footer but no header. I started to look around the first signature occurrence and noted that there was a kind of table:

Filesystem hex dump

Unfortunately, I wasn't able to find a relation bewteen this table and the scattered files content.

Just a color note, I found that Firmware Mod Kit is using the same header struct for MemFS file systems mentioned on the IOActive's post and devttys0's post, however, that's not entirely my case.

Getting the binary being loaded by IDA

Once I extracted the files, I needed a way to continue with the analysis because the files I obtained weren't too interesting and I was sure that there were more inside. As I already mentioned, they weren't, probably, files with all their entire structure but parts of them handled by some respective service (HTTP, FTP, etc;), a kind of standalone server handling different types of protocols and sending chunks of files.

I needed a way to load the extracted data in IDA and get the all the functions, strings, symbols, etc; in order to do a more extensive analysis. The main goal I needed to accomplish was to find the base address to load the binary.

Again, I found two interesting and helpful blog posts.

In the post from the devttys0's blog, the author obtains the base address by analyzing an extracted MIPS binary with symbols extracted from the firmware in IDA. That binary is responsible for loading and decompressing a LZMA stream located at certain part of the firmware. The base address to load the decompressed stream is clearly referenced in the code. It's also mentioned that this address is used at some code found on the Internet:

So it appears that the LZMA data that we extracted earlier does contain executable code, and that it is decompressed and loaded into memory at address 0x80001000. A Google search for 'vxworks lzmadecode' turns up some source code that confirms this conclusion.

The referenced link doesn't work for me. I did a quick search again and found only one reference to a LZMA decoder using that address.

In the post from cq674350529's blog, the base address to load the binary is obtained from the output provided by Binwalk. A U-Boot header is found in the firmware file and contains a reference to an entry point address, that entry point address is the one used as base address in IDA.

In my case, both approaches didn't fit with my situation. I didn't have any U-Boot header or anything similar to that and I didn't have any other binary to analyze. My guess is that, in my case, the base address doesn't need to be inside the firmware data becase, probably, it's hardcoded in the bootloader. All the code needed to unpack the data is located on the device. What now? well, there is a coincidence in both post, the base address: 0x80001000. Let's try with that:

IDA disassembly memory organization, second attempt

This was the result:

IDA disassembly, second attempt

Bingo! After hitting C to create code in the disassembly window, IDA started to work with the auto-analysis and found a lot of functions. Several more than the one found when no base address was specified :P

Now, it can be seen that some of the strings that are part of chuncked files, are referenced in the code:

IDA xrefs

OK, I was able to reverse engineer the binary but, even though I had some string xrefs, there weren't any symbol. In the case of the cq674350529's blog post, the symbols were scattered in the firmware file and author needed to write a script in order to build the symbol table because IDA wasn't able to do it with the auto-analysis. However, in my case, I didn't have any known VxWorks symbol.

Anyway, by using the string xrefs and numeric constants it's possible to reverse engineer and rename functions. For example, I found the following string xref Error: MD4Update MD already done.. That string belongs to the Implementation of MD4 Message Digest Algorithm. It's possible to find some of the functions referenced in the source code by searching for the initialization constants:

#define I0  0x67452301       /* Initial values for MD buffer */
#define I1  0xefcdab89
#define I2  0x98badcfe
#define I3  0x10325476
#define C2  013240474631     /* round 2 constant = sqrt(2) in octal */
#define C3  015666365641     /* round 3 constant = sqrt(3) in octal */

This is the corresponding code in the MIPS binary:

lui     $a1, 0xEFCD
lui     $a2, 0x98BA
lui     $v0, 0x1032
ori     $v1, 0x2301
li      $a1, 0xEFCDAB89 --> this constant
li      $a2, 0x98BADCFE --> this one too
li      $v0, 0x10325476
sw      $v0, 0xC($a0)
sw      $v1, 0($a0)
sw      $a1, 4($a0)
sw      $a2, 8($a0)
sw      $zero, 0x10($a0)
jr      $ra
sw      $zero, 0x14($a0)

That's the MD4Init() function.

Event though this is a good and common approach when reversing a binary without symbols, it takes a considerable time to have a decent list of reversed functions. Also, there are some common functions such as printf, memset, memcpy, etc... that can be easily identified but sometimes it depends on the compiler optimizations and flags used during the compilation process. So, I was sure there was an easier way to automate the work of identifying common functions and I was right :)

Getting symbols

In order to automate the task of identify common functions I used Rizzo, an IDA plugin developed by devttys0 that includes fuzzy signatures made with unique string references and other metrics as identifiable actions, such as memory accesses, references to constant values, and function calls. However, the plugin is a little bit outdated, the last change was 2 years ago in order to support IDA 7 and only supports Python 2. I reported an issue and @fuzzywalls (thanks btw) sent me his fork. However, that fork didn't work with the build of IDA 7.4 I had (191112).

After some work, I was finally able to fix all the issues and make it work with my IDA build and Python 2. These are the results:

Found 3 formal matches in 0.00 seconds.
Found 107 fuzzy matches in 0.00 seconds.
Found 188 string matches in 0.00 seconds.
Found 40 immediate matches in 0.00 seconds.
Renamed 162 functions in 0.54 seconds.
Signatures applied in 24.04 seconds
Caching 'Functions window'... ok
Caching 'Names window'... ok
IDA functions table

You can find my version of Rizzo here.

Last words

Once I got some symbols with Rizzo, I conducted a session to search for vulnerabilities but it wasn't successful :(

I started looking at the functions that Rizzo was able to recognize.

Some of the functions are related to DES and MD4 and weren't too interesting for me, these functions belong to some public implementation, for example: Implementation of MD4 Message Digest Algorithm. However, there are strings referencing functions that are, I guess, related to the configuration file (the configuration file can be dumped from the router's GUI, it's encrypted/obfuscated in some way):

ROM:801FA99C aApcfgencryptio:.ascii "apCfgEncryptionSet(%d, TRUE) failed\r\n"<0>
ROM:801FA9C2                 .byte    0
ROM:801FA9C3                 .byte    0
ROM:801FA9C4 aIwirelessprivs:.ascii "iWirelessPrivSelect = %d, secSubType[1] = %d\r\n"<0>
ROM:801FA9F3                 .byte    0
ROM:801FA9F4 aApcfgauthtypes:.ascii "apCfgAuthTypeSet() failed\r\n"<0>
ROM:801FAA10 aApcfgcipherset:.ascii "apCfgCipherSet() failed\r\n"<0>
ROM:801FAA2A                 .byte    0
ROM:801FAA2B                 .byte    0
ROM:801FAA2C aApcfgkeysrcset:.ascii "apCfgKeySrcSet() failed\r\n"<0>
ROM:801FAA46                 .byte    0
ROM:801FAA47                 .byte    0
ROM:801FAA48 aApcfggroupkeyu:.ascii "apCfgGroupKeyUpdateIntervalSet() failed\r\n"<0>
ROM:801FAA72                 .byte    0
ROM:801FAA73                 .byte    0
ROM:801FAA74 aErrorUnableToG:.ascii "ERROR: Unable to generate key with given passphrase and ssid\n"
ROM:801FAA74                 .ascii <0>
ROM:801FAAB2                 .byte    0
ROM:801FAAB3                 .byte    0
ROM:801FAAB4 aApcfgpassphras:.ascii "apCfgPassphraseSet() failed\r\n"<0>
ROM:801FAAD2                 .byte    0
ROM:801FAAD3                 .byte    0
ROM:801FAAD4 aApcfgpassphras_0:.ascii "apCfgPassphraseKeySet() failed\r\n"<0>

Unfortunately, there aren't xrefs to these strings.

Some others like the following, took me to this source code (apparently, related to Ethernet and a kind of descriptors conversion between Atheros and VxWorks):

ROM:801FA5B8 aVxdesctomblkEr:.ascii "\n"
ROM:801FA5B8                 .ascii "vxDescToMblk: error in join mBlk\n"<0>
ROM:801FA5DB                 .byte    0
ROM:801FA5DC aVxmblktodescMP:.ascii "vxMblkToDesc: M_PKTHDR not set on first mblk\n"<0>

Again, these strings don't have xrefs :(

The rest of the functions don't look so promising (as can be seen in this file), most of them are standard functions or auxiliary. There isn't any interesting function like a custom handler or something similar.

Another thing I tried was using the Burp Suite in order to intercept some of the requests from the HTTP server (the only service exposed by the device) and try to trigger a crash in some way, for example, modifying the parameters of the request. The following example is a capture from the NTP server configuration:

GET /userRpm/DateTimeCfgRpm.htm?timezone=540&month=1&day=1&year=2006&hour=11&minute=59&second=52&ntpA=0.0.0.0&ntpB=0.0.0.0&Submit=Save HTTP/1.1

Host: 192.168.1.1

User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.5

Accept-Encoding: gzip, deflate

Referer: http://192.168.1.1/userRpm/DateTimeCfgRpm.htm

Authorization: Basic bnJpdmE6SncyWGtEYWpWbG1jaA==

Connection: close

Upgrade-Insecure-Requests: 1

In order to get some output, I connected a serial device interface on the other side in order to catch for possible outputs, however, Windows 10 was giving me some troubles with the USB interface. I got some errors related to max voltage consumption and had to restart the PC several times in order to continue.

At the end, by doing some test and looking at the code in the binary, it seems that everything is handeld there (in a single binary) and most answers are harcoded. The server responds with hardcoded "chunks" of data and any other request that doesn't fit with the answer, is dropped. Different requests (with different parameters) return the same answer, for example, when changing the NTP server.

What else can be done?

Well, probably a debugger inside the device would have been very helpful. A possibility to accomplish this task would have been to modify the firmware and upload a custom GDB server. However, it would have required to compile GDB server for MIPS using the compilation tools for VxWorks (apparently, it's only targeting x86/x64/ARM and VxWorks 7). If someone feels like trying, please, I'd be really glad to hear from them!.

Conclusion

Certainly, to reverse engineer an unknown architecture or deal with an uncommon OS has its complications but sometimes, even with few information available, it's possible to make some progress and finish the work. In this particular case, it was possible thanks to some guys who found similar issues in the past and documented the process.

Thanks to that, I was able to reverse engineer the unknown file system used in the router and to get familiar with the operating system. It would have been nice to find a vulnerability but I wasn't lucky at this time. Begin able to gain access to a shell or to upload tools to the router in order to debug it, would really be a big step towards assessing its security and hunting for its vulnerabilities, and will require some more work!

Anyway, I hope that sharing this experience will in turn prove useful to others.

Acknowledgements

  • Thanks to all my Quarkslab colleagues who proofread this article and provided valuable feedback.

  • Thanks to Craig Heffner (devttys0) for his amazing posts, tools and continuous work on embedded devices.

  • Thanks to Evan Walls (fuzzywalls) for helping me with his Rizzo fork. I just fixed what was needed to make it work instead of porting the entire plugin to the new IDA 7.4 API.


If you would like to learn more about our security audits and explore how we can help you, get in touch with us!