Migration (#120)
Go mod tidy [skip ci] Add modules/migration to docs [skip ci] update date fmt Merge branch 'master' into feature/migration # Conflicts: # pkg/routes/api/v1/info.go Add docs on how to create a migrator Add available migrators to /info endpoint Return a message once everything was migrated successfully Add swagger docs for wunderlist migration Docs for migration [skip ci] Fix due date fixture in migration test Fix staticcheck Fix lint Logging and cleanup Make the migrator work with real data Add routes for migration Fix misspell Add method to store a full vikunja structure into vikunja Add getting all data from wunderlist Add attachment migration from wunderlist Add done and done at to wunderlist migrator Add todo Add wunderlist auth url implementation Fix lint Finish wunderlist migration Added all structs for the wunderlist migratior Fix owner field being null for user shared namespaces (#119) Update copyright year (#118) Add option to disable registration (#117) Added migrator interface Co-authored-by: kolaente <k@knt.li> Reviewed-on: https://kolaente.dev/vikunja/api/pulls/120
This commit is contained in:
konrad
parent
1d2cdf0fb8
commit
9e39399689
19 changed files with 1451 additions and 11 deletions
|
|
@ -117,3 +117,19 @@ files:
|
|||
# The maximum size of a file, as a human-readable string.
|
||||
# Warning: The max size is limited 2^64-1 bytes due to the underlying datatype
|
||||
maxsize: 20MB
|
||||
|
||||
migration:
|
||||
# These are the settings for the wunderlist migrator
|
||||
wunderlist:
|
||||
# Wheter to enable the wunderlist migrator or not
|
||||
enable: true
|
||||
# The client id, required for making requests to the wunderlist api
|
||||
# You need to register your vikunja instance at https://developer.wunderlist.com/apps/new to get this
|
||||
clientid:
|
||||
# The client secret, also required for making requests to the wunderlist api
|
||||
clientsecret:
|
||||
# The url where clients are redirected after they authorized Vikunja to access their wunderlist stuff.
|
||||
# This needs to match the url you entered when registering your Vikunja instance at wunderlist.
|
||||
# This is usually the frontend url where the frontend then makes a request to /migration/wunderlist/migrate
|
||||
# with the code obtained from the wunderlist api.
|
||||
redirecturl:
|
||||
|
|
|
|||
91
docs/content/doc/development/migration.md
Normal file
91
docs/content/doc/development/migration.md
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
---
|
||||
date: "2020-01-19:16:00+02:00"
|
||||
title: "Migrations"
|
||||
draft: false
|
||||
type: "doc"
|
||||
menu:
|
||||
sidebar:
|
||||
parent: "development"
|
||||
---
|
||||
|
||||
# Writing a migrator for Vikunja
|
||||
|
||||
It is possible to migrate data from other to-do services to Vikunja.
|
||||
To make this easier, we have put together a few helpers which are documented on this page.
|
||||
|
||||
In general, each migrator implements a migrator interface which is then called from a client.
|
||||
The interface makes it possible to use helper methods which handle http an focus only on the implementation of the migrator itself.
|
||||
|
||||
### Structure
|
||||
|
||||
All migrator implementations live in their own package in `pkg/modules/migration/<name-of-the-service>`.
|
||||
When creating a new migrator, you should place all related code inside that module.
|
||||
|
||||
### Migrator interface
|
||||
|
||||
The migrator interface is defined as follows:
|
||||
|
||||
```go
|
||||
// Migrator is the basic migrator interface which is shared among all migrators
|
||||
type Migrator interface {
|
||||
// Migrate is the interface used to migrate a user's tasks from another platform to vikunja.
|
||||
// The user object is the user who's tasks will be migrated.
|
||||
Migrate(user *models.User) error
|
||||
// AuthURL returns a url for clients to authenticate against.
|
||||
// The use case for this are Oauth flows, where the server token should remain hidden and not
|
||||
// known to the frontend.
|
||||
AuthURL() string
|
||||
}
|
||||
```
|
||||
|
||||
### Defining http routes
|
||||
|
||||
Once your migrator implements the migration interface, it becomes possible to use the helper http handlers.
|
||||
Their usage is very similar to the [general web handler](https://kolaente.dev/vikunja/web#user-content-defining-routes-using-the-standard-web-handler):
|
||||
|
||||
```go
|
||||
// This is an example for the Wunderlist migrator
|
||||
if config.MigrationWunderlistEnable.GetBool() {
|
||||
wunderlistMigrationHandler := &migrationHandler.MigrationWeb{
|
||||
MigrationStruct: func() migration.Migrator {
|
||||
return &wunderlist.Migration{}
|
||||
},
|
||||
}
|
||||
m.GET("/wunderlist/auth", wunderlistMigrationHandler.AuthURL)
|
||||
m.POST("/wunderlist/migrate", wunderlistMigrationHandler.Migrate)
|
||||
}
|
||||
```
|
||||
|
||||
You should also document the routes with [swagger annotations]({{< ref "../practical-instructions/swagger-docs.md" >}}).
|
||||
|
||||
### Insertion helper method
|
||||
|
||||
There is a method available in the `migration` package which takes a fully nested Vikunja structure and creates it with all relations.
|
||||
This means you start by adding a namespace, then add lists inside of that namespace, then tasks in the lists and so on.
|
||||
|
||||
The root structure must be present as `[]*models.NamespaceWithLists`.
|
||||
|
||||
Then call the method like so:
|
||||
|
||||
```go
|
||||
fullVikunjaHierachie, err := convertWunderlistToVikunja(wContent)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
err = migration.InsertFromStructure(fullVikunjaHierachie, user)
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
You should add at least an option to enable or disable the migration.
|
||||
Chances are, you'll need some more options for things like client ID and secret
|
||||
(if the other service uses oAuth as an authentication flow).
|
||||
|
||||
The easiest way to implement an on/off switch is to check whether your migration service is enabled or not when
|
||||
registering the routes, and then simply don't registering the routes in the case it is disabled.
|
||||
|
||||
#### Making the migrator public in `/info`
|
||||
|
||||
You should make your migrator available in the `/info` endpoint so that frontends can display options to enable them or not.
|
||||
To do this, add an entry to `pkg/routes/api/v1/info.go`.
|
||||
|
|
@ -22,6 +22,10 @@ In general, this api repo has the following structure:
|
|||
* `metrics`
|
||||
* `migration`
|
||||
* `models`
|
||||
* `modules`
|
||||
* `migration`
|
||||
* `handler`
|
||||
* `wunderlist`
|
||||
* `red`
|
||||
* `routes`
|
||||
* `api/v1`
|
||||
|
|
@ -85,7 +89,7 @@ To learn how it works and how to add new metrics, take a look at [how metrics wo
|
|||
This package handles all migrations.
|
||||
All migrations are stored and executed here.
|
||||
|
||||
To learn more, take a look at the [migrations docs]({{< ref "../development/migrations.md">}}).
|
||||
To learn more, take a look at the [migrations docs]({{< ref "../development/db-migrations.md">}}).
|
||||
|
||||
### models
|
||||
|
||||
|
|
@ -97,6 +101,12 @@ Because this package is pretty huge, there are several documents and how-to's ab
|
|||
* [Adding a feature]({{< ref "../practical-instructions/feature.md">}})
|
||||
* [Making calls to the database]({{< ref "../practical-instructions/database.md">}})
|
||||
|
||||
### modules
|
||||
|
||||
#### migration
|
||||
|
||||
See [writing a migrator]({{< ref "migration.md" >}}).
|
||||
|
||||
### red (redis)
|
||||
|
||||
This package initializes a connection to a redis server.
|
||||
|
|
|
|||
|
|
@ -160,4 +160,20 @@ files:
|
|||
# The maximum size of a file, as a human-readable string.
|
||||
# Warning: The max size is limited 2^64-1 bytes due to the underlying datatype
|
||||
maxsize: 20MB
|
||||
|
||||
migration:
|
||||
# These are the settings for the wunderlist migrator
|
||||
wunderlist:
|
||||
# Wheter to enable the wunderlist migrator or not
|
||||
enable: true
|
||||
# The client id, required for making requests to the wunderlist api
|
||||
# You need to register your vikunja instance at https://developer.wunderlist.com/apps/new to get this
|
||||
clientid:
|
||||
# The client secret, also required for making requests to the wunderlist api
|
||||
clientsecret:
|
||||
# The url where clients are redirected after they authorized Vikunja to access their wunderlist stuff.
|
||||
# This needs to match the url you entered when registering your Vikunja instance at wunderlist.
|
||||
# This is usually the frontend url where the frontend then makes a request to /migration/wunderlist/migrate
|
||||
# with the code obtained from the wunderlist api.
|
||||
redirecturl:
|
||||
{{< /highlight >}}
|
||||
|
|
|
|||
1
go.mod
1
go.mod
|
|
@ -59,7 +59,6 @@ require (
|
|||
github.com/onsi/gomega v1.4.3 // indirect
|
||||
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7
|
||||
github.com/pelletier/go-toml v1.4.0 // indirect
|
||||
github.com/pkg/errors v0.8.1 // indirect
|
||||
github.com/prometheus/client_golang v0.9.2
|
||||
github.com/samedi/caldav-go v3.0.0+incompatible
|
||||
github.com/shurcooL/httpfs v0.0.0-20190527155220-6a4d4a70508b
|
||||
|
|
|
|||
|
|
@ -89,6 +89,11 @@ const (
|
|||
|
||||
FilesBasePath Key = `files.basepath`
|
||||
FilesMaxSize Key = `files.maxsize`
|
||||
|
||||
MigrationWunderlistEnable Key = `migration.wunderlist.enable`
|
||||
MigrationWunderlistClientID Key = `migration.wunderlist.clientid`
|
||||
MigrationWunderlistClientSecret Key = `migration.wunderlist.clientsecret`
|
||||
MigrationWunderlistRedirectURL Key = `migration.wunderlist.redirecturl`
|
||||
)
|
||||
|
||||
// GetString returns a string config value
|
||||
|
|
|
|||
|
|
@ -39,6 +39,9 @@ type File struct {
|
|||
CreatedByID int64 `xorm:"int(11) not null" json:"-"`
|
||||
|
||||
File afero.File `xorm:"-" json:"-"`
|
||||
// This ReadCloser is only used for migration purposes. Use with care!
|
||||
// There is currentlc no better way of doing this.
|
||||
FileContent []byte `xorm:"-" json:"-"`
|
||||
}
|
||||
|
||||
// TableName is the table name for the files table
|
||||
|
|
|
|||
97
pkg/modules/migration/create_from_structure.go
Normal file
97
pkg/modules/migration/create_from_structure.go
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
// Vikunja is a todo-list application to facilitate your life.
|
||||
// Copyright 2018-2020 Vikunja and contributors. All rights reserved.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
package migration
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"code.vikunja.io/api/pkg/models"
|
||||
"io/ioutil"
|
||||
)
|
||||
|
||||
// InsertFromStructure takes a fully nested Vikunja data structure and a user and then creates everything for this user
|
||||
// (Namespaces, tasks, etc. Even attachments and relations.)
|
||||
func InsertFromStructure(str []*models.NamespaceWithLists, user *models.User) (err error) {
|
||||
|
||||
// Create all namespaces
|
||||
for _, n := range str {
|
||||
err = n.Create(user)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Create all lists
|
||||
for _, l := range n.Lists {
|
||||
// The tasks slice is going to be reset during the creation of the list so we rescue it here to be able
|
||||
// to still loop over the tasks aftere the list was created.
|
||||
tasks := l.Tasks
|
||||
|
||||
l.NamespaceID = n.ID
|
||||
err = l.Create(user)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Create all tasks
|
||||
for _, t := range tasks {
|
||||
t.ListID = l.ID
|
||||
err = t.Create(user)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Create all relation for each task
|
||||
for kind, tasks := range t.RelatedTasks {
|
||||
// First create the related tasks if they does not exist
|
||||
for _, rt := range tasks {
|
||||
if rt.ID == 0 {
|
||||
err = rt.Create(user)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Then create the relation
|
||||
taskRel := &models.TaskRelation{
|
||||
TaskID: rt.ID,
|
||||
OtherTaskID: t.ID,
|
||||
RelationKind: kind,
|
||||
}
|
||||
err = taskRel.Create(user)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Create all attachments for each task
|
||||
for _, a := range t.Attachments {
|
||||
// Check if we have a file to create
|
||||
if len(a.File.FileContent) > 0 {
|
||||
a.TaskID = t.ID
|
||||
fr := ioutil.NopCloser(bytes.NewReader(a.File.FileContent))
|
||||
err = a.NewAttachment(fr, a.File.Name, a.File.Size, user)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
66
pkg/modules/migration/handler/handler.go
Normal file
66
pkg/modules/migration/handler/handler.go
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
// Vikunja is a todo-list application to facilitate your life.
|
||||
// Copyright 2018-2020 Vikunja and contributors. All rights reserved.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
package handler
|
||||
|
||||
import (
|
||||
"code.vikunja.io/api/pkg/models"
|
||||
"code.vikunja.io/api/pkg/modules/migration"
|
||||
"code.vikunja.io/web/handler"
|
||||
"github.com/labstack/echo/v4"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// MigrationWeb holds the web migration handler
|
||||
type MigrationWeb struct {
|
||||
MigrationStruct func() migration.Migrator
|
||||
}
|
||||
|
||||
// AuthURL is returned to the user when requesting the auth url
|
||||
type AuthURL struct {
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
// AuthURL is the web handler to get the auth url
|
||||
func (mw *MigrationWeb) AuthURL(c echo.Context) error {
|
||||
ms := mw.MigrationStruct()
|
||||
return c.JSON(http.StatusOK, &AuthURL{URL: ms.AuthURL()})
|
||||
}
|
||||
|
||||
// Migrate calls the migration method
|
||||
func (mw *MigrationWeb) Migrate(c echo.Context) error {
|
||||
ms := mw.MigrationStruct()
|
||||
|
||||
// Get the user from context
|
||||
user, err := models.GetCurrentUser(c)
|
||||
if err != nil {
|
||||
return handler.HandleHTTPError(err, c)
|
||||
}
|
||||
|
||||
// Bind user request stuff
|
||||
err = c.Bind(ms)
|
||||
if err != nil {
|
||||
return echo.NewHTTPError(http.StatusBadRequest, "No or invalid model provided: "+err.Error())
|
||||
}
|
||||
|
||||
// Do the migration
|
||||
err = ms.Migrate(user)
|
||||
if err != nil {
|
||||
return handler.HandleHTTPError(err, c)
|
||||
}
|
||||
|
||||
return c.JSON(http.StatusOK, models.Message{Message: "Everything was migrated successfully."})
|
||||
}
|
||||
31
pkg/modules/migration/migrator.go
Normal file
31
pkg/modules/migration/migrator.go
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
// Copyright 2019 Vikunja and contriubtors. All rights reserved.
|
||||
//
|
||||
// This file is part of Vikunja.
|
||||
//
|
||||
// Vikunja is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// Vikunja is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Vikunja. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
package migration
|
||||
|
||||
import "code.vikunja.io/api/pkg/models"
|
||||
|
||||
// Migrator is the basic migrator interface which is shared among all migrators
|
||||
type Migrator interface {
|
||||
// Migrate is the interface used to migrate a user's tasks from another platform to vikunja.
|
||||
// The user object is the user who's tasks will be migrated.
|
||||
Migrate(user *models.User) error
|
||||
// AuthURL returns a url for clients to authenticate against.
|
||||
// The use case for this are Oauth flows, where the server token should remain hidden and not
|
||||
// known to the frontend.
|
||||
AuthURL() string
|
||||
}
|
||||
BIN
pkg/modules/migration/wunderlist/testimage.jpg
Normal file
BIN
pkg/modules/migration/wunderlist/testimage.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.4 KiB |
482
pkg/modules/migration/wunderlist/wunderlist.go
Normal file
482
pkg/modules/migration/wunderlist/wunderlist.go
Normal file
|
|
@ -0,0 +1,482 @@
|
|||
// Vikunja is a todo-list application to facilitate your life.
|
||||
// Copyright 2018-2020 Vikunja and contributors. All rights reserved.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
package wunderlist
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"code.vikunja.io/api/pkg/config"
|
||||
"code.vikunja.io/api/pkg/files"
|
||||
"code.vikunja.io/api/pkg/log"
|
||||
"code.vikunja.io/api/pkg/models"
|
||||
"code.vikunja.io/api/pkg/modules/migration"
|
||||
"code.vikunja.io/api/pkg/utils"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Migration represents the implementation of the migration for wunderlist
|
||||
type Migration struct {
|
||||
// Code is the code used to get a user api token
|
||||
Code string `query:"code" json:"code"`
|
||||
}
|
||||
|
||||
// This represents all necessary fields for getting an api token for the wunderlist api from a code
|
||||
type wunderlistAuthRequest struct {
|
||||
ClientID string `json:"client_id"`
|
||||
ClientSecret string `json:"client_secret"`
|
||||
Code string `json:"code"`
|
||||
}
|
||||
|
||||
type wunderlistAuthToken struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
}
|
||||
|
||||
type task struct {
|
||||
ID int `json:"id"`
|
||||
AssigneeID int `json:"assignee_id"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
CreatedByID int `json:"created_by_id"`
|
||||
DueDate string `json:"due_date"`
|
||||
ListID int `json:"list_id"`
|
||||
Revision int `json:"revision"`
|
||||
Starred bool `json:"starred"`
|
||||
Title string `json:"title"`
|
||||
Completed bool `json:"completed"`
|
||||
CompletedAt time.Time `json:"completed_at"`
|
||||
}
|
||||
|
||||
type list struct {
|
||||
ID int `json:"id"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Title string `json:"title"`
|
||||
ListType string `json:"list_type"`
|
||||
Type string `json:"type"`
|
||||
Revision int `json:"revision"`
|
||||
|
||||
Migrated bool `json:"-"`
|
||||
}
|
||||
|
||||
type folder struct {
|
||||
ID int `json:"id"`
|
||||
Title string `json:"title"`
|
||||
ListIds []int `json:"list_ids"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
CreatedByRequestID string `json:"created_by_request_id"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Type string `json:"type"`
|
||||
Revision int `json:"revision"`
|
||||
}
|
||||
|
||||
type note struct {
|
||||
ID int `json:"id"`
|
||||
TaskID int `json:"task_id"`
|
||||
Content string `json:"content"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Revision int `json:"revision"`
|
||||
}
|
||||
|
||||
type file struct {
|
||||
ID int `json:"id"`
|
||||
URL string `json:"url"`
|
||||
TaskID int `json:"task_id"`
|
||||
ListID int `json:"list_id"`
|
||||
UserID int `json:"user_id"`
|
||||
FileName string `json:"file_name"`
|
||||
ContentType string `json:"content_type"`
|
||||
FileSize int `json:"file_size"`
|
||||
LocalCreatedAt time.Time `json:"local_created_at"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Type string `json:"type"`
|
||||
Revision int `json:"revision"`
|
||||
}
|
||||
|
||||
type reminder struct {
|
||||
ID int `json:"id"`
|
||||
Date time.Time `json:"date"`
|
||||
TaskID int `json:"task_id"`
|
||||
Revision int `json:"revision"`
|
||||
Type string `json:"type"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type subtask struct {
|
||||
ID int `json:"id"`
|
||||
TaskID int `json:"task_id"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
CreatedByID int `json:"created_by_id"`
|
||||
Revision int `json:"revision"`
|
||||
Title string `json:"title"`
|
||||
}
|
||||
|
||||
type wunderlistContents struct {
|
||||
tasks []*task
|
||||
lists []*list
|
||||
folders []*folder
|
||||
notes []*note
|
||||
files []*file
|
||||
reminders []*reminder
|
||||
subtasks []*subtask
|
||||
}
|
||||
|
||||
func convertListForFolder(listID int, list *list, content *wunderlistContents) (*models.List, error) {
|
||||
|
||||
l := &models.List{
|
||||
Title: list.Title,
|
||||
Created: list.CreatedAt.Unix(),
|
||||
}
|
||||
|
||||
// Find all tasks belonging to this list and put them in
|
||||
for _, t := range content.tasks {
|
||||
if t.ListID == listID {
|
||||
newTask := &models.Task{
|
||||
Text: t.Title,
|
||||
Created: t.CreatedAt.Unix(),
|
||||
Done: t.Completed,
|
||||
}
|
||||
|
||||
// Set Done At
|
||||
if newTask.Done {
|
||||
newTask.DoneAtUnix = t.CompletedAt.Unix()
|
||||
}
|
||||
|
||||
// Parse the due date
|
||||
if t.DueDate != "" {
|
||||
dueDate, err := time.Parse("2006-01-02", t.DueDate)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
newTask.DueDateUnix = dueDate.Unix()
|
||||
}
|
||||
|
||||
// Find related notes
|
||||
for _, n := range content.notes {
|
||||
if n.TaskID == t.ID {
|
||||
newTask.Description = n.Content
|
||||
}
|
||||
}
|
||||
|
||||
// Attachments
|
||||
for _, f := range content.files {
|
||||
if f.TaskID == t.ID {
|
||||
// Download the attachment and put it in the file
|
||||
resp, err := http.Get(f.URL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
buf := &bytes.Buffer{}
|
||||
_, err = buf.ReadFrom(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
newTask.Attachments = append(newTask.Attachments, &models.TaskAttachment{
|
||||
File: &files.File{
|
||||
Name: f.FileName,
|
||||
Mime: f.ContentType,
|
||||
Size: uint64(f.FileSize),
|
||||
Created: f.CreatedAt,
|
||||
CreatedUnix: f.CreatedAt.Unix(),
|
||||
// We directly pass the file contents here to have a way to link the attachment to the file later.
|
||||
// Because we don't have an ID for our task at this point of the migration, we cannot just throw all
|
||||
// attachments in a slice and do the work of downloading and properly storing them later.
|
||||
FileContent: buf.Bytes(),
|
||||
},
|
||||
Created: f.CreatedAt.Unix(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Subtasks
|
||||
for _, s := range content.subtasks {
|
||||
if s.TaskID == t.ID {
|
||||
if newTask.RelatedTasks[models.RelationKindSubtask] == nil {
|
||||
newTask.RelatedTasks = make(models.RelatedTaskMap)
|
||||
}
|
||||
newTask.RelatedTasks[models.RelationKindSubtask] = append(newTask.RelatedTasks[models.RelationKindSubtask], &models.Task{
|
||||
Text: s.Title,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Reminders
|
||||
for _, r := range content.reminders {
|
||||
if r.TaskID == t.ID {
|
||||
newTask.RemindersUnix = append(newTask.RemindersUnix, r.Date.Unix())
|
||||
}
|
||||
}
|
||||
|
||||
l.Tasks = append(l.Tasks, newTask)
|
||||
}
|
||||
}
|
||||
return l, nil
|
||||
}
|
||||
|
||||
func convertWunderlistToVikunja(content *wunderlistContents) (fullVikunjaHierachie []*models.NamespaceWithLists, err error) {
|
||||
|
||||
// Make a map from the list with the key being list id for easier handling
|
||||
listMap := make(map[int]*list, len(content.lists))
|
||||
for _, l := range content.lists {
|
||||
listMap[l.ID] = l
|
||||
}
|
||||
|
||||
// First, we look through all folders and create namespaces for them.
|
||||
for _, folder := range content.folders {
|
||||
namespace := &models.NamespaceWithLists{
|
||||
Namespace: models.Namespace{
|
||||
Name: folder.Title,
|
||||
Created: folder.CreatedAt.Unix(),
|
||||
Updated: folder.UpdatedAt.Unix(),
|
||||
},
|
||||
}
|
||||
|
||||
// Then find all lists for that folder
|
||||
for _, listID := range folder.ListIds {
|
||||
if list, exists := listMap[listID]; exists {
|
||||
l, err := convertListForFolder(listID, list, content)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
namespace.Lists = append(namespace.Lists, l)
|
||||
// And mark the list as migrated so we don't iterate over it again
|
||||
list.Migrated = true
|
||||
}
|
||||
}
|
||||
|
||||
// And then finally put the namespace (which now has all the details) back in the full array.
|
||||
fullVikunjaHierachie = append(fullVikunjaHierachie, namespace)
|
||||
}
|
||||
|
||||
// At the end, loop over all lists which don't belong to a namespace and put them in a default namespace
|
||||
if len(listMap) > 0 {
|
||||
newNamespace := &models.NamespaceWithLists{
|
||||
Namespace: models.Namespace{
|
||||
Name: "Migrated from wunderlist",
|
||||
},
|
||||
}
|
||||
|
||||
for _, list := range listMap {
|
||||
|
||||
if list.Migrated {
|
||||
continue
|
||||
}
|
||||
|
||||
l, err := convertListForFolder(list.ID, list, content)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
newNamespace.Lists = append(newNamespace.Lists, l)
|
||||
}
|
||||
|
||||
fullVikunjaHierachie = append(fullVikunjaHierachie, newNamespace)
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
func makeAuthGetRequest(token *wunderlistAuthToken, urlPart string, v interface{}, urlParams url.Values) error {
|
||||
req, err := http.NewRequest(http.MethodGet, "https://a.wunderlist.com/api/v1/"+urlPart, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("X-Access-Token", token.AccessToken)
|
||||
req.Header.Set("X-Client-ID", config.MigrationWunderlistClientID.GetString())
|
||||
req.URL.RawQuery = urlParams.Encode()
|
||||
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
_, err = buf.ReadFrom(resp.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if resp.StatusCode > 399 {
|
||||
return fmt.Errorf("wunderlist API Error: Status Code: %d, Response was: %s", resp.StatusCode, buf.String())
|
||||
}
|
||||
|
||||
// If the response is an empty json array, we need to exit here, otherwise this breaks the json parser since it
|
||||
// expects a null for an empty slice
|
||||
str := buf.String()
|
||||
if str == "[]" {
|
||||
return nil
|
||||
}
|
||||
|
||||
return json.Unmarshal(buf.Bytes(), v)
|
||||
}
|
||||
|
||||
// Migrate migrates a user's wunderlist lists, tasks, etc.
|
||||
// @Summary Migrate all lists, tasks etc. from wunderlist
|
||||
// @Description Migrates all folders, lists, tasks, notes, reminders, subtasks and files from wunderlist to vikunja.
|
||||
// @tags migration
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Security JWTKeyAuth
|
||||
// @Param migrationCode body wunderlist.Migration true "The auth code previously obtained from the auth url. See the docs for /migration/wunderlist/auth."
|
||||
// @Success 200 {object} models.Message "A message telling you everything was migrated successfully."
|
||||
// @Failure 500 {object} models.Message "Internal server error"
|
||||
// @Router /migration/wunderlist/migrate [post]
|
||||
func (w *Migration) Migrate(user *models.User) (err error) {
|
||||
|
||||
log.Debugf("[Wunderlist migration] Starting wunderlist migration for user %d", user.ID)
|
||||
|
||||
// Struct init
|
||||
wContent := &wunderlistContents{
|
||||
tasks: []*task{},
|
||||
lists: []*list{},
|
||||
folders: []*folder{},
|
||||
notes: []*note{},
|
||||
files: []*file{},
|
||||
reminders: []*reminder{},
|
||||
subtasks: []*subtask{},
|
||||
}
|
||||
|
||||
// 0. Get api token from oauth user token
|
||||
authRequest := wunderlistAuthRequest{
|
||||
ClientID: config.MigrationWunderlistClientID.GetString(),
|
||||
ClientSecret: config.MigrationWunderlistClientSecret.GetString(),
|
||||
Code: w.Code,
|
||||
}
|
||||
jsonAuth, err := json.Marshal(authRequest)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
resp, err := http.Post("https://www.wunderlist.com/oauth/access_token", "application/json", bytes.NewBuffer(jsonAuth))
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
authToken := &wunderlistAuthToken{}
|
||||
err = json.NewDecoder(resp.Body).Decode(authToken)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
log.Debugf("[Wunderlist migration] Start getting all data from wunderlist for user %d", user.ID)
|
||||
|
||||
// 1. Get all folders
|
||||
err = makeAuthGetRequest(authToken, "folders", &wContent.folders, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
// 2. Get all lists
|
||||
err = makeAuthGetRequest(authToken, "lists", &wContent.lists, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
for _, l := range wContent.lists {
|
||||
|
||||
listQueryParam := url.Values{"list_id": []string{strconv.Itoa(l.ID)}}
|
||||
|
||||
// 3. Get all tasks for each list
|
||||
tasks := []*task{}
|
||||
err = makeAuthGetRequest(authToken, "tasks", &tasks, listQueryParam)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
wContent.tasks = append(wContent.tasks, tasks...)
|
||||
|
||||
// 3. Get all done tasks for each list
|
||||
doneTasks := []*task{}
|
||||
err = makeAuthGetRequest(authToken, "tasks", &doneTasks, url.Values{"list_id": []string{strconv.Itoa(l.ID)}, "completed": []string{"true"}})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
wContent.tasks = append(wContent.tasks, doneTasks...)
|
||||
|
||||
// 4. Get all notes for all lists
|
||||
notes := []*note{}
|
||||
err = makeAuthGetRequest(authToken, "notes", ¬es, listQueryParam)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
wContent.notes = append(wContent.notes, notes...)
|
||||
|
||||
// 5. Get all files for all lists
|
||||
fils := []*file{}
|
||||
err = makeAuthGetRequest(authToken, "files", &fils, listQueryParam)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
wContent.files = append(wContent.files, fils...)
|
||||
|
||||
// 6. Get all reminders for all lists
|
||||
reminders := []*reminder{}
|
||||
err = makeAuthGetRequest(authToken, "reminders", &reminders, listQueryParam)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
wContent.reminders = append(wContent.reminders, reminders...)
|
||||
|
||||
// 7. Get all subtasks for all lists
|
||||
subtasks := []*subtask{}
|
||||
err = makeAuthGetRequest(authToken, "subtasks", &subtasks, listQueryParam)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
wContent.subtasks = append(wContent.subtasks, subtasks...)
|
||||
}
|
||||
|
||||
log.Debugf("[Wunderlist migration] Got all data from wunderlist for user %d", user.ID)
|
||||
log.Debugf("[Wunderlist migration] Migrating data to vikunja format for user %d", user.ID)
|
||||
|
||||
// Convert + Insert everything
|
||||
fullVikunjaHierachie, err := convertWunderlistToVikunja(wContent)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
log.Debugf("[Wunderlist migration] Done migrating data to vikunja format for user %d", user.ID)
|
||||
log.Debugf("[Wunderlist migration] Insert data into db for user %d", user.ID)
|
||||
|
||||
err = migration.InsertFromStructure(fullVikunjaHierachie, user)
|
||||
|
||||
log.Debugf("[Wunderlist migration] Done inserting data into db for user %d", user.ID)
|
||||
log.Debugf("[Wunderlist migration] Wunderlist migration for user %d done", user.ID)
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// AuthURL returns the url users need to authenticate against
|
||||
// @Summary Get the auth url from wunderlist
|
||||
// @Description Returns the auth url where the user needs to get its auth code. This code can then be used to migrate everything from wunderlist to Vikunja.
|
||||
// @tags migration
|
||||
// @Produce json
|
||||
// @Security JWTKeyAuth
|
||||
// @Success 200 {object} handler.AuthURL "The auth url."
|
||||
// @Failure 500 {object} models.Message "Internal server error"
|
||||
// @Router /migration/wunderlist/auth [get]
|
||||
func (w *Migration) AuthURL() string {
|
||||
return "https://www.wunderlist.com/oauth/authorize?client_id=" +
|
||||
config.MigrationWunderlistClientID.GetString() +
|
||||
"&redirect_uri=" +
|
||||
config.MigrationWunderlistRedirectURL.GetString() +
|
||||
"&state=" + utils.MakeRandomString(32)
|
||||
}
|
||||
352
pkg/modules/migration/wunderlist/wunderlist_test.go
Normal file
352
pkg/modules/migration/wunderlist/wunderlist_test.go
Normal file
|
|
@ -0,0 +1,352 @@
|
|||
// Vikunja is a todo-list application to facilitate your life.
|
||||
// Copyright 2018-2020 Vikunja and contributors. All rights reserved.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
package wunderlist
|
||||
|
||||
import (
|
||||
"code.vikunja.io/api/pkg/config"
|
||||
"code.vikunja.io/api/pkg/files"
|
||||
"code.vikunja.io/api/pkg/models"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"gopkg.in/d4l3k/messagediff.v1"
|
||||
"io/ioutil"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestWunderlistParsing(t *testing.T) {
|
||||
|
||||
config.InitConfig()
|
||||
|
||||
time1, err := time.Parse(time.RFC3339Nano, "2013-08-30T08:29:46.203Z")
|
||||
assert.NoError(t, err)
|
||||
time2, err := time.Parse(time.RFC3339Nano, "2013-08-30T08:36:13.273Z")
|
||||
assert.NoError(t, err)
|
||||
time3, err := time.Parse(time.RFC3339Nano, "2013-09-05T08:36:13.273Z")
|
||||
assert.NoError(t, err)
|
||||
time4, err := time.Parse(time.RFC3339Nano, "2013-08-02T11:58:55Z")
|
||||
assert.NoError(t, err)
|
||||
|
||||
exampleFile, err := ioutil.ReadFile(config.ServiceRootpath.GetString() + "/pkg/modules/migration/wunderlist/testimage.jpg")
|
||||
assert.NoError(t, err)
|
||||
|
||||
createTestTask := func(id, listID int, done bool) *task {
|
||||
completedAt, err := time.Parse(time.RFC3339Nano, "1970-01-01T00:00:00Z")
|
||||
assert.NoError(t, err)
|
||||
if done {
|
||||
completedAt = time1
|
||||
}
|
||||
return &task{
|
||||
ID: id,
|
||||
AssigneeID: 123,
|
||||
CreatedAt: time1,
|
||||
DueDate: "2013-09-05",
|
||||
ListID: listID,
|
||||
Title: "Ipsum" + strconv.Itoa(id),
|
||||
Completed: done,
|
||||
CompletedAt: completedAt,
|
||||
}
|
||||
}
|
||||
|
||||
createTestNote := func(id, taskID int) *note {
|
||||
return ¬e{
|
||||
ID: id,
|
||||
TaskID: taskID,
|
||||
Content: "Lorem Ipsum dolor sit amet",
|
||||
CreatedAt: time3,
|
||||
UpdatedAt: time2,
|
||||
}
|
||||
}
|
||||
|
||||
fixtures := &wunderlistContents{
|
||||
folders: []*folder{
|
||||
{
|
||||
ID: 123,
|
||||
Title: "Lorem Ipsum",
|
||||
ListIds: []int{1, 2, 3, 4},
|
||||
CreatedAt: time1,
|
||||
UpdatedAt: time2,
|
||||
},
|
||||
},
|
||||
lists: []*list{
|
||||
{
|
||||
ID: 1,
|
||||
CreatedAt: time1,
|
||||
Title: "Lorem1",
|
||||
},
|
||||
{
|
||||
ID: 2,
|
||||
CreatedAt: time1,
|
||||
Title: "Lorem2",
|
||||
},
|
||||
{
|
||||
ID: 3,
|
||||
CreatedAt: time1,
|
||||
Title: "Lorem3",
|
||||
},
|
||||
{
|
||||
ID: 4,
|
||||
CreatedAt: time1,
|
||||
Title: "Lorem4",
|
||||
},
|
||||
{
|
||||
ID: 5,
|
||||
CreatedAt: time4,
|
||||
Title: "List without a namespace",
|
||||
},
|
||||
},
|
||||
tasks: []*task{
|
||||
createTestTask(1, 1, false),
|
||||
createTestTask(2, 1, false),
|
||||
createTestTask(3, 2, true),
|
||||
createTestTask(4, 2, false),
|
||||
createTestTask(5, 3, false),
|
||||
createTestTask(6, 3, true),
|
||||
createTestTask(7, 3, true),
|
||||
createTestTask(8, 3, false),
|
||||
createTestTask(9, 4, true),
|
||||
createTestTask(10, 4, true),
|
||||
},
|
||||
notes: []*note{
|
||||
createTestNote(1, 1),
|
||||
createTestNote(2, 2),
|
||||
createTestNote(3, 3),
|
||||
},
|
||||
files: []*file{
|
||||
{
|
||||
ID: 1,
|
||||
URL: "https://vikunja.io/testimage.jpg", // Using an image which we are hosting, so it'll still be up
|
||||
TaskID: 1,
|
||||
ListID: 1,
|
||||
FileName: "file.md",
|
||||
ContentType: "text/plain",
|
||||
FileSize: 12345,
|
||||
CreatedAt: time2,
|
||||
UpdatedAt: time4,
|
||||
},
|
||||
{
|
||||
ID: 2,
|
||||
URL: "https://vikunja.io/testimage.jpg",
|
||||
TaskID: 3,
|
||||
ListID: 2,
|
||||
FileName: "file2.md",
|
||||
ContentType: "text/plain",
|
||||
FileSize: 12345,
|
||||
CreatedAt: time3,
|
||||
UpdatedAt: time4,
|
||||
},
|
||||
},
|
||||
reminders: []*reminder{
|
||||
{
|
||||
ID: 1,
|
||||
Date: time4,
|
||||
TaskID: 1,
|
||||
CreatedAt: time4,
|
||||
UpdatedAt: time4,
|
||||
},
|
||||
{
|
||||
ID: 2,
|
||||
Date: time3,
|
||||
TaskID: 4,
|
||||
CreatedAt: time3,
|
||||
UpdatedAt: time3,
|
||||
},
|
||||
},
|
||||
subtasks: []*subtask{
|
||||
{
|
||||
ID: 1,
|
||||
TaskID: 2,
|
||||
CreatedAt: time4,
|
||||
Title: "LoremSub1",
|
||||
},
|
||||
{
|
||||
ID: 2,
|
||||
TaskID: 2,
|
||||
CreatedAt: time4,
|
||||
Title: "LoremSub2",
|
||||
},
|
||||
{
|
||||
ID: 3,
|
||||
TaskID: 4,
|
||||
CreatedAt: time4,
|
||||
Title: "LoremSub3",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
expectedHierachie := []*models.NamespaceWithLists{
|
||||
{
|
||||
Namespace: models.Namespace{
|
||||
Name: "Lorem Ipsum",
|
||||
Created: time1.Unix(),
|
||||
Updated: time2.Unix(),
|
||||
},
|
||||
Lists: []*models.List{
|
||||
{
|
||||
Created: time1.Unix(),
|
||||
Title: "Lorem1",
|
||||
Tasks: []*models.Task{
|
||||
{
|
||||
Text: "Ipsum1",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Description: "Lorem Ipsum dolor sit amet",
|
||||
Attachments: []*models.TaskAttachment{
|
||||
{
|
||||
File: &files.File{
|
||||
Name: "file.md",
|
||||
Mime: "text/plain",
|
||||
Size: 12345,
|
||||
Created: time2,
|
||||
CreatedUnix: time2.Unix(),
|
||||
FileContent: exampleFile,
|
||||
},
|
||||
Created: time2.Unix(),
|
||||
},
|
||||
},
|
||||
RemindersUnix: []int64{time4.Unix()},
|
||||
},
|
||||
{
|
||||
Text: "Ipsum2",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Description: "Lorem Ipsum dolor sit amet",
|
||||
RelatedTasks: map[models.RelationKind][]*models.Task{
|
||||
models.RelationKindSubtask: {
|
||||
{
|
||||
Text: "LoremSub1",
|
||||
},
|
||||
{
|
||||
Text: "LoremSub2",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Created: time1.Unix(),
|
||||
Title: "Lorem2",
|
||||
Tasks: []*models.Task{
|
||||
{
|
||||
Text: "Ipsum3",
|
||||
Done: true,
|
||||
DoneAtUnix: time1.Unix(),
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Description: "Lorem Ipsum dolor sit amet",
|
||||
Attachments: []*models.TaskAttachment{
|
||||
{
|
||||
File: &files.File{
|
||||
Name: "file2.md",
|
||||
Mime: "text/plain",
|
||||
Size: 12345,
|
||||
Created: time3,
|
||||
CreatedUnix: time3.Unix(),
|
||||
FileContent: exampleFile,
|
||||
},
|
||||
Created: time3.Unix(),
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Text: "Ipsum4",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
RemindersUnix: []int64{time3.Unix()},
|
||||
RelatedTasks: map[models.RelationKind][]*models.Task{
|
||||
models.RelationKindSubtask: {
|
||||
{
|
||||
Text: "LoremSub3",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Created: time1.Unix(),
|
||||
Title: "Lorem3",
|
||||
Tasks: []*models.Task{
|
||||
{
|
||||
Text: "Ipsum5",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
},
|
||||
{
|
||||
Text: "Ipsum6",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Done: true,
|
||||
DoneAtUnix: time1.Unix(),
|
||||
},
|
||||
{
|
||||
Text: "Ipsum7",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Done: true,
|
||||
DoneAtUnix: time1.Unix(),
|
||||
},
|
||||
{
|
||||
Text: "Ipsum8",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Created: time1.Unix(),
|
||||
Title: "Lorem4",
|
||||
Tasks: []*models.Task{
|
||||
{
|
||||
Text: "Ipsum9",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Done: true,
|
||||
DoneAtUnix: time1.Unix(),
|
||||
},
|
||||
{
|
||||
Text: "Ipsum10",
|
||||
DueDateUnix: 1378339200,
|
||||
Created: time1.Unix(),
|
||||
Done: true,
|
||||
DoneAtUnix: time1.Unix(),
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Namespace: models.Namespace{
|
||||
Name: "Migrated from wunderlist",
|
||||
},
|
||||
Lists: []*models.List{
|
||||
{
|
||||
Created: time4.Unix(),
|
||||
Title: "List without a namespace",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
hierachie, err := convertWunderlistToVikunja(fixtures)
|
||||
assert.NoError(t, err)
|
||||
assert.NotNil(t, hierachie)
|
||||
if diff, equal := messagediff.PrettyDiff(hierachie, expectedHierachie); !equal {
|
||||
t.Errorf("ListUser.ReadAll() = %v, want %v, diff: %v", hierachie, expectedHierachie, diff)
|
||||
}
|
||||
}
|
||||
|
|
@ -30,6 +30,7 @@ type vikunjaInfos struct {
|
|||
LinkSharingEnabled bool `json:"link_sharing_enabled"`
|
||||
MaxFileSize string `json:"max_file_size"`
|
||||
RegistrationEnabled bool `json:"registration_enabled"`
|
||||
AvailableMigrators []string `json:"available_migrators"`
|
||||
}
|
||||
|
||||
// Info is the handler to get infos about this vikunja instance
|
||||
|
|
@ -40,12 +41,16 @@ type vikunjaInfos struct {
|
|||
// @Success 200 {object} v1.vikunjaInfos
|
||||
// @Router /info [get]
|
||||
func Info(c echo.Context) error {
|
||||
return c.JSON(http.StatusOK, vikunjaInfos{
|
||||
infos := vikunjaInfos{
|
||||
Version: version.Version,
|
||||
FrontendURL: config.ServiceFrontendurl.GetString(),
|
||||
Motd: config.ServiceMotd.GetString(),
|
||||
LinkSharingEnabled: config.ServiceEnableLinkSharing.GetBool(),
|
||||
MaxFileSize: config.FilesMaxSize.GetString(),
|
||||
RegistrationEnabled: config.ServiceEnableRegistration.GetBool(),
|
||||
})
|
||||
}
|
||||
if config.MigrationWunderlistEnable.GetBool() {
|
||||
infos.AvailableMigrators = append(infos.AvailableMigrators, "wunderlist")
|
||||
}
|
||||
return c.JSON(http.StatusOK, infos)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -47,6 +47,9 @@ import (
|
|||
"code.vikunja.io/api/pkg/config"
|
||||
"code.vikunja.io/api/pkg/log"
|
||||
"code.vikunja.io/api/pkg/models"
|
||||
"code.vikunja.io/api/pkg/modules/migration"
|
||||
migrationHandler "code.vikunja.io/api/pkg/modules/migration/handler"
|
||||
"code.vikunja.io/api/pkg/modules/migration/wunderlist"
|
||||
apiv1 "code.vikunja.io/api/pkg/routes/api/v1"
|
||||
"code.vikunja.io/api/pkg/routes/caldav"
|
||||
_ "code.vikunja.io/api/pkg/swagger" // To generate swagger docs
|
||||
|
|
@ -372,6 +375,20 @@ func registerAPIRoutes(a *echo.Group) {
|
|||
}
|
||||
a.PUT("/teams/:team/members", teamMemberHandler.CreateWeb)
|
||||
a.DELETE("/teams/:team/members/:user", teamMemberHandler.DeleteWeb)
|
||||
|
||||
// Migrations
|
||||
m := a.Group("/migration")
|
||||
|
||||
// Wunderlist
|
||||
if config.MigrationWunderlistEnable.GetBool() {
|
||||
wunderlistMigrationHandler := &migrationHandler.MigrationWeb{
|
||||
MigrationStruct: func() migration.Migrator {
|
||||
return &wunderlist.Migration{}
|
||||
},
|
||||
}
|
||||
m.GET("/wunderlist/auth", wunderlistMigrationHandler.AuthURL)
|
||||
m.POST("/wunderlist/migrate", wunderlistMigrationHandler.Migrate)
|
||||
}
|
||||
}
|
||||
|
||||
func registerCalDavRoutes(c *echo.Group) {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
// GENERATED BY THE COMMAND ABOVE; DO NOT EDIT
|
||||
// This file was generated by swaggo/swag at
|
||||
// 2019-12-07 22:54:02.661375666 +0100 CET m=+0.164990732
|
||||
// 2020-01-19 16:18:04.294790395 +0100 CET m=+0.176548843
|
||||
|
||||
package swagger
|
||||
|
||||
|
|
@ -1615,6 +1615,83 @@ var doc = `{
|
|||
}
|
||||
}
|
||||
},
|
||||
"/migration/wunderlist/auth": {
|
||||
"get": {
|
||||
"security": [
|
||||
{
|
||||
"JWTKeyAuth": []
|
||||
}
|
||||
],
|
||||
"description": "Returns the auth url where the user needs to get its auth code. This code can then be used to migrate everything from wunderlist to Vikunja.",
|
||||
"produces": [
|
||||
"application/json"
|
||||
],
|
||||
"tags": [
|
||||
"migration"
|
||||
],
|
||||
"summary": "Get the auth url from wunderlist",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "The auth url.",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/handler.AuthURL"
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal server error",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/models.Message"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/migration/wunderlist/migrate": {
|
||||
"post": {
|
||||
"security": [
|
||||
{
|
||||
"JWTKeyAuth": []
|
||||
}
|
||||
],
|
||||
"description": "Migrates all folders, lists, tasks, notes, reminders, subtasks and files from wunderlist to vikunja.",
|
||||
"consumes": [
|
||||
"application/json"
|
||||
],
|
||||
"produces": [
|
||||
"application/json"
|
||||
],
|
||||
"tags": [
|
||||
"migration"
|
||||
],
|
||||
"summary": "Migrate all lists, tasks etc. from wunderlist",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "The auth code previously obtained from the auth url. See the docs for /migration/wunderlist/auth.",
|
||||
"name": "migrationCode",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"$ref": "#/definitions/wunderlist.Migration"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "A message telling you everything was migrated successfully.",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/models.Message"
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal server error",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/models.Message"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/namespace/{id}": {
|
||||
"post": {
|
||||
"security": [
|
||||
|
|
@ -4463,6 +4540,14 @@ var doc = `{
|
|||
}
|
||||
}
|
||||
},
|
||||
"handler.AuthURL": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"models.APIUserPassword": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
|
|
@ -5527,6 +5612,15 @@ var doc = `{
|
|||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"wunderlist.Migration": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"code": {
|
||||
"description": "Code is the code used to get a user api token",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"securityDefinitions": {
|
||||
|
|
|
|||
|
|
@ -1597,6 +1597,83 @@
|
|||
}
|
||||
}
|
||||
},
|
||||
"/migration/wunderlist/auth": {
|
||||
"get": {
|
||||
"security": [
|
||||
{
|
||||
"JWTKeyAuth": []
|
||||
}
|
||||
],
|
||||
"description": "Returns the auth url where the user needs to get its auth code. This code can then be used to migrate everything from wunderlist to Vikunja.",
|
||||
"produces": [
|
||||
"application/json"
|
||||
],
|
||||
"tags": [
|
||||
"migration"
|
||||
],
|
||||
"summary": "Get the auth url from wunderlist",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "The auth url.",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/handler.AuthURL"
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal server error",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/models.Message"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/migration/wunderlist/migrate": {
|
||||
"post": {
|
||||
"security": [
|
||||
{
|
||||
"JWTKeyAuth": []
|
||||
}
|
||||
],
|
||||
"description": "Migrates all folders, lists, tasks, notes, reminders, subtasks and files from wunderlist to vikunja.",
|
||||
"consumes": [
|
||||
"application/json"
|
||||
],
|
||||
"produces": [
|
||||
"application/json"
|
||||
],
|
||||
"tags": [
|
||||
"migration"
|
||||
],
|
||||
"summary": "Migrate all lists, tasks etc. from wunderlist",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "The auth code previously obtained from the auth url. See the docs for /migration/wunderlist/auth.",
|
||||
"name": "migrationCode",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"$ref": "#/definitions/wunderlist.Migration"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "A message telling you everything was migrated successfully.",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/models.Message"
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal server error",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/models.Message"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/namespace/{id}": {
|
||||
"post": {
|
||||
"security": [
|
||||
|
|
@ -4444,6 +4521,14 @@
|
|||
}
|
||||
}
|
||||
},
|
||||
"handler.AuthURL": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"models.APIUserPassword": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
|
|
@ -5508,6 +5593,15 @@
|
|||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"wunderlist.Migration": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"code": {
|
||||
"description": "Code is the code used to get a user api token",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"securityDefinitions": {
|
||||
|
|
|
|||
|
|
@ -13,6 +13,11 @@ definitions:
|
|||
size:
|
||||
type: integer
|
||||
type: object
|
||||
handler.AuthURL:
|
||||
properties:
|
||||
url:
|
||||
type: string
|
||||
type: object
|
||||
models.APIUserPassword:
|
||||
properties:
|
||||
email:
|
||||
|
|
@ -860,6 +865,12 @@ definitions:
|
|||
version:
|
||||
type: string
|
||||
type: object
|
||||
wunderlist.Migration:
|
||||
properties:
|
||||
code:
|
||||
description: Code is the code used to get a user api token
|
||||
type: string
|
||||
type: object
|
||||
info:
|
||||
contact:
|
||||
email: hello@vikunja.io
|
||||
|
|
@ -1935,6 +1946,57 @@ paths:
|
|||
summary: Login
|
||||
tags:
|
||||
- user
|
||||
/migration/wunderlist/auth:
|
||||
get:
|
||||
description: Returns the auth url where the user needs to get its auth code.
|
||||
This code can then be used to migrate everything from wunderlist to Vikunja.
|
||||
produces:
|
||||
- application/json
|
||||
responses:
|
||||
"200":
|
||||
description: The auth url.
|
||||
schema:
|
||||
$ref: '#/definitions/handler.AuthURL'
|
||||
"500":
|
||||
description: Internal server error
|
||||
schema:
|
||||
$ref: '#/definitions/models.Message'
|
||||
security:
|
||||
- JWTKeyAuth: []
|
||||
summary: Get the auth url from wunderlist
|
||||
tags:
|
||||
- migration
|
||||
/migration/wunderlist/migrate:
|
||||
post:
|
||||
consumes:
|
||||
- application/json
|
||||
description: Migrates all folders, lists, tasks, notes, reminders, subtasks
|
||||
and files from wunderlist to vikunja.
|
||||
parameters:
|
||||
- description: The auth code previously obtained from the auth url. See the
|
||||
docs for /migration/wunderlist/auth.
|
||||
in: body
|
||||
name: migrationCode
|
||||
required: true
|
||||
schema:
|
||||
$ref: '#/definitions/wunderlist.Migration'
|
||||
type: object
|
||||
produces:
|
||||
- application/json
|
||||
responses:
|
||||
"200":
|
||||
description: A message telling you everything was migrated successfully.
|
||||
schema:
|
||||
$ref: '#/definitions/models.Message'
|
||||
"500":
|
||||
description: Internal server error
|
||||
schema:
|
||||
$ref: '#/definitions/models.Message'
|
||||
security:
|
||||
- JWTKeyAuth: []
|
||||
summary: Migrate all lists, tasks etc. from wunderlist
|
||||
tags:
|
||||
- migration
|
||||
/namespace/{id}:
|
||||
post:
|
||||
consumes:
|
||||
|
|
|
|||
Loading…
Reference in a new issue