Jan
17

Getting to your PiDrive

ocipv6I wrote few times about my PiDrive already, this is continuation of the work in progress and I would like to share what I did since the last time.

Getting accessible

We need to address two problems regarding the accessibility of PiDrive. First one is actually not that you need to access your PiDrive from Internet, but something much simpler. Once you connect your PiDrive to your local network, you need to find out it’s local address first so you can set it up. There are various options, for example including avahi or netbios and configuring them to publish some recognizable name. I’m sure everybody has those in mind and I do as well. But I wanted to start first with something that might have escaped the others and what I consider quite simple but at the same time quite effective approach. On boot, I display ownCloud logo on HDMI attached display if there is one and bellow it address of the device. My PiDrive came with 90 degrees angle HDMI converter so it looked like it is expected that you will connect display to it. And reading what is written on HDMI is much simpler and reliable than anything you do on computer.

Other accessibility issue is getting to your PiDrive from Internet. I already prepared kinda solution (still need to implement tunnel option thought) for the Internet of tomorrow (IPv6), but as quite some people still live in the past, I extended my application and now it supports even UPnP. What it is and how does it work? If you have smart enough router that allows such kind of thing (most of them, although you probably need to enable it), you can instruct your PiDrive to open up a port on it and forward it to itself. Tricky part is that your router has to support it and you kinda need public IPv4 on the other side (otherwise it misses the point). So it doesn’t solve everything, but gives PiDrive accessible on IPv4 to at least some people. And Dynv6 I implemented previously while playing with IPv6 should be able to resolve to your public IPv4 as well. So you can get ready for the future, while still maintaining compatibility for people living in stone age.

Image

Both of the improvements mentioned above are installed into my image and also finally installed ownCloud on it. Although currently just a git snapshot. For final version, I’ll have to switch to packages I guess, but currently I have some dependency issues with them (and php7) that I need to solve first. You can now try improvements I mentioned by yourself. Just inspect my GitHub repo or you can still download my temporal binaries (now updated).

Jan
10

My way of booting PiDrive (Raspberry Pi2)

PiDrive bootingSome time ago I started playing with PiDrive project. I implemented an application that I think will be useful to people using it in the end – some simple IPv6 enabler/browser and DynDNS client. But I kinda cheated and implemented it on the ARM board I already had at home. Over the last week I didn’t had much free time, but I still continued on the project and I got my Pi booting my custom image. How will PiDrive boot was subject to lengthy discussions on mailing list, so I wanted to provide a proof of concept of how do I think it can be done. As it is long post, TLDR version at the end 😉

Starting points

First, let’s take a look at how Raspberry boots. Most ARM board I encountered so far used u-Boot that was either in internal flash memory or on the SD card at some predefined address. Raspberry searches for a files with predefined names on FAT filesystem that it expects to find on first partition of your SD card. On one hand, it looks weirder, on the other it is simpler if you don’t have any prior experience with ARM. And if you do, you can still put u-Boot there to get all the options u-Boot provides.

Other observation to take into account made by others is that SD cards die quickly in those devices. Mine didn’t yet, but I don’t do any crazy stuff and tend to be lucky. But let’s take for granted that during the life of the device, SD card might die and maybe even multiple times. We should somehow address that. And recovering from broken SD card should be simple so even average Joe can do that.

As I’m openSUSE guy, I obviously picked openSUSE to power my PiDrive, but nothing I did should depend on chosen distribution. I picked 13.2 as Leap for armv7 is not ready yet (about that some other time) and I added few packages that I think can be useful on OBS in separate project.

Last point I have is that I would personally prefer to have system installed on SD card. It will be slower to start, but it will let harddrive to spin down and go to sleep when not used. Prolonging it’s life and reducing the noise of spinning drive. But if system is running from SD card, second paragraph is even more important – we need to gracefully handle SD cards death.

How I boot my Pi

Now you know from what I started and let’s take a look with what I ended up. I wanted to make firsts time installation as easy as possible. As SD cards come preformatted to FAT already, all you have to do to boot is just to copy some files there. I found out what files I need there, tweaked a configuration a little bit and compiled my own kernel. Why? To compile all drivers needed during boot directly in and to include custom initramdisk. For those who don’t know, initramdisk is, as name suggests, small disk in RAM that contains few programs and script that is executed before the system boots and can be used if you need to do some complicated stuff before booting. And that is exactly what I had in mind.

On first boot, my initramdisk takes everything on SD card, copies it to memory (we have 1G of RAM and pretty much nothing is running at this point), then it repartitions the SD card and copies the files back. It makes first partition with FAT smaller and creates new Ext2 partition behind it. What are those for? First partition contains stuff needed by the board to boot up (various firmwares and kernel) but also squashfs compressed rootfs. Once init script will discover correct ext2 partition, it will use it as overlay on top of this squashfs image to make OS image writeable and boots the system using this overlayed filesystem. During boot it also checks whether overlay is on SD is empty (for example because card died) and if so, looks for the backup on harddrive and if found, it is used to populate overlay. So the idea is to use SD card, but backup everything and make a recovery in case SD card dies as easy as possible – you just get a new card from nearest convenience store, unpack an archive on it and boot up. Can’t be simpler I guess.

Little about why choosing squashfs and overlayfs on top of it. SD card is pretty small, and having compressed filesystem will make it much easier to fit. And also, I played with it some time ago and thanks to compressed rootfs device can actually read from it faster as SD cards are slow, but decompression is easy and fast.

If you want to see it or want to try my approach, I put sources to my GitHub repo. It has some implicit dependencies I was too lazy to enumerate and I should write some documentation, but it should download kernel and busybox sources, compile everything, download openSUSE JeOS and Raspberry binary blobs, put everything together and produce SD directory with files to put on SD card. Just for convenience, I will temporally provide binary archive that you can just unpack to SD card and test. Currently it is really just booting. No ownCloud installed yet, although nginx and php7 are there. It should get IP via DHCP, use HDMI out and you should be able to either ssh to it or login on second console using root account and password owncloud.

Plans

What is currently just in TODO is to actually install and play with ownCloud. There is plenty of questions to discuss, like where to put the database and which one. In case of MySQL, we could even divide it and put some tables there and others elsewhere. We have SD, HDD and RAM. Speaking about HDD, also in TODO is to find, format and mount USB drive. That’s also about customizing the final rootfs. Last thing that I would like to do at some point is to try to have an easy way how to recompress the system after updates/additional installs and cleanup the overlay. But that is far future. This post is mainly for people playing with PiDrive to explain which way I’m going and pointing everybody to my GitHub repo :-)

TLDR version

I got my Raspberry Pi 2 booting from SD card and automatically repartitioning it and it’s easy to deploy, just unpack this archive on SD card with FAT. Sources are at my GitHub, so you can take a look at what it does. No ownCloud installed yet. Root password is owncloud.

Dec
31

Getting IPv6 for your ownCloud

As you could have read, I joined the ownClouds PiDrive effort. I like the idea and we were brainstorming on the mailing list regarding what can we do. One notion really popped out. If you have ownCloud at home, you might be interested in reaching your home cloud from anywhere you go. And if you don’t have public IPv4 or you don’t want to forward public ports from the router, you might be interested in getting IPv6 for you home cloud. It can be pretty easy. Both on your home cloud and your notebook. I would like to talk about few options I considered and how and which I decided to integrate them (also into ownCloud app that you can use anywhere).

Overview of options

Native IPv6

You might be lucky and get IPv6 segment directly from your Internet provider. I know mine does offer it. But I also know that quite some providers are trying to fight inevitable future and keep postponing IPv6 deployment. On the other hand, I heard that there are some providers that migrated fully to IPv6 and no longer offer IPv4, just NAT64. If you have native IPv6, you probably know about it and if your router is correctly setup, your home NAS as well as your other computers will get it automatically. So for those lucky ones, my app will just detect IPv6 and display the address you can use to connect to your cloud.

6to4

Kinda nice way to get IPv6, but requires public IPv4 and some advanced setup on the device that has this IP. So probably nothing average Joe will do. But if he does and propagate it to his PiDrive, app will detect it and show it.

Tunnel from IPv6 broker

This is quite popular option. You register with some tunnel broker (like HE or SixXS) and they will give you fixed IPv6 address or range of IPv6 address that you can use however you want. You get always the same IP no matter where you are and setup is usually pretty easy, the only tricky part is registration which often requires you to fill in quite some personal details. I was thinking about providing this as an option on PiDrive, but the need to register somewhere and need to choose a broker sounded quite bothersome, so I decided to support mainly the last option I’m going to mention which I consider the most user-friendly. But as in previous case, if you set this up, the app will detect that IP address that you got this way and display it and this option might be added later on as I consider it quite useful.

Teredo

Teredo is I would say the easiest way (from the end-users point of view) how to get IPv6. You just need to run the client on your machine, and it will figure everything out and assign you some IPv6 address that you can use. It works behind NAT and you don’t have to know anything, it will figure out everything – even the closest relay to use. There are some disadvantages. There is an overhead with figuring out everything, the protocol itself has some overhead and on top of that, your IP depends on public IP of your NAT and how it handles your traffic. It also depends on port you were assigned by your NAT, so your IP will likely change with every reboot. But overall I think it is viable solution for end users if used together with some DynDNS service (where you would need to register but mostly with less personal info). I have in my TODO to add option to support DynDNS as well (started working on it already, but really just started, so nothing published yet).

Conclusion

I think making it easy for home users to enable IPv6 on their home cloud and educating people about how to get IPv6 on their other machines is probably the best way to go regarding how to let people have their data available everywhere they go. An I hope the app I’m working on will help to achieve that.

Dec
29

PiDrive unboxing

Not so long ago ownCloud announced their cooperation with Western Digital. Outcome is PiDrive – basically home NAS solution. ARM board (RaspberryPi2) connected with HDD. And with the announcement of the cooperation came a challenge. Community was asked to come up with ideas regarding what would they do with it. Who was interested in working on the image that will be shipped as part of the final solution was offered a prototype of the device. I was one of the guys brainstorming about what to do with it. I had some ideas and already started working on some of them. More about them and the progress later. Currently I want to share some pictures of the PiDrive (as I already received the prototype) – obligatory unboxing and few thoughts on hardware.

Important note – whole this blog post is about prototype. Final device can be totally different.

So what do we have here? Let’s start with the ARM board. Raspberry Pi 2 has some advantages and some disadvantages from specs alone (just opened up the box, not booted up yet). It has only USB 2.0 (while drive itself supports USB 3.0) and only 100MBit ethernet, which is not that much nowadays. With BananaPi, it could get harddrive attached directly via SATA port and it would have 1GBit ethernet. But on the other hand 4 cores can be quite useful on device that is expected to run webserver. And if we learned anything from RaspberryPi it is that marketing matters a lot and thanks to it, there will be huge community around Raspberry Pi2 and thus plenty of interesting projects (and peripherals) can later come to PiDrive.

Box I got from post office

Box I got from post office
Picture 1 of 9

That kinda brings us to the box. As you can see on pictures, the prototype actually come with two boxes. One is black and other whiteish. Both of them are translucent, so you can see when some LED on your Pi is on or when harddrive is doing something. Both cases looks the same except of the colour. Really great idea is how to handle all the cables that go out of Pi. As you can guess, some are pretty mandatory – like power and ethernet. But there can be plenty of optional ones. More USB devices (keyboard, mouse, …), HDMI, some GPIO attached devices, maybe more. There would have to be plenty of holes to support all of them. The box solves the problem by letting you plug everything in and then having long and narrow hole in the back where you can guide out as many cables as you want. I really like the design of the box. Except one thing. I kinda miss the top of the box. In my case, top is open. You can see inside which is nice on one hand, but dust will be falling in quite heavily. I hope this will be fixed in final version or maybe just my devkit was missing the closure. One other small issue I have with box is that the stand that holds Pi and drive (board to board) doesn’t have hard plastic everywhere between them and I’m kinda worried that both board could touch if they vibrate enough and short out, but maybe it’s well tested and impossible and I’m just worried without any reason. But despite this criticism, I really like the boxing for the PiDrive. And I will somehow create a top of the box myself.

One last thing I haven’t mentioned yet is hard drive itself. It is 2.5″ 1T WD drive. More specs once I get the device booting. And there is a really cool cable attached to it. It has one power input, one power output as micro-USB (that goes to Raspberry), one USB that goes to USB port of Raspberry to connect it with drive and one micro-USB 3 that goes to the drive (both power and data). Kinda cool how it connects power and data while having four heads.

Overall I really like the idea and the project. There is a lot of work that needs to be done, but I’m happy that I’ll be part of the effort. So take a look at the pictures for now and I will create another blog post once I’ll have something up and running and some practical experience with the setup.

Dec
28

openSUSE Summit Asia 2015

Me & BudhaLast year first ever openSUSE Summit Asia took place in Beijing. From all the reports it sounded really awesome and I regretted that I couldn’t go. This year, I was lucky enough to manage to go. I was selected to do a board keynote and I got some of my travel expenses sponsored by Travel Support Programme (big thanks!). So how was this years openSUSE Summit Asia from my point of view? In short, amazing :-) In long, read on…

Let’s start with my Taiwan trip. I have never been to Taiwan before and visiting this country, which is quite different from my homeland, was by itself amazing. Lot of small shops everywhere, old and new temples between modern shopping centers, scooters everywhere, free WiFi in turistic places in Taipei… And friendly people everywhere. One really handy thing I noticed is that on every underground station, some buses and on few other places as well are recharging stations for your notebooks/cellphones. It is only starting to happen in my home town and in past I had several occasions when I could really use something like that. In Taipei, quite often you can see people recharging their devices so they can continue travelling :-) Other really interesting thing is kinda Foursquare predecessor. On plenty of places, you have a rubber stamp that you can use to put a mark in your log. On underground stations, all tourist offices, tourist attractions and more. So as you are travelling, you have a log of places you have been to. Really interesting comparing to all the high-tech stuff you see all around. I could go on how amazing Taiwan was from tourists point of view, but lets get to the main reason I was there – openSUSE Summit Asia.

Me & MascotEverything started on Friday by openSUSE Leap release party. It was combined with Ubuntu party. Both communities mixed well together and it was obvious that they know each other well and are friends. As everywhere in Taiwan, everybody was really friendly. I met there guy from Canonical who had Ubuntu phone, I showed him my Jolla, he showed me his Ubuntu phone, we talked and I was giving it some thoughts. I was thinking about what to buy as my next phone as Jolla was in deep troubles back than. Luckily they are now out of the dark, so I can look forward to next Jolla phone, but Ubuntu phone is still interesting especially as it can be probably converted into openSUSE phone :-) There was some talking in Chinese by the host that everybody seemed to enjoy and then lightning talks. One Debian guy did a presentation of C web framework. After that I stepped up and did presentation of my favourite C++ web framework afterwards. We had some pizza, some local food, something to drink and even birthday celebration of one of the community members. It was a lot of fun.

During the weekend was the summit itself. There was plenty of talks and workshops, some of it in Chinese, some of it in English. I did the keynote on behalf of openSUSE Board. Than I tried attending the Chinese workshop, but I got lost quite quickly, so I ended up following only the English part of the track. Regardless, there were some really interesting talks. One I would like to stress out is the talk that emphasises the importance of summit in Asia. There was a talk and long discussion regarding input methods. I found out that input methods varies a lot. Some uses English transcription, some maps keys on English keyboard to some Chinese signs, not sure whether full characters. But it varies a lot and it is not that easy to set it up. Now it is easier thanks to Chameleon tongue :-) And it is a type of problem, that is not that interesting in Europe/America, but is really important in Asia.

Apart from talks, there was as on every conference a lot of talks in the corridor and connecting with people and making new friends. We even had a visit from lizardy mascot :-) I learned what to visit on Taiwan, where to buy stuff, how is the conference infrastructure setup, that there is ongoing effort to publish openSUSE Japanese magazine and much more. I was a lot of fun. As a proof of how much fun was it, you can take a look at the pictures taken during the conference.

Older posts «