I'd like to build my own Graph Database, where should I start?

Perhaps this book will help you.

The Practitioner’s Guide to Graph Data