23 Commits

Author SHA1 Message Date
Lukas F. Hartmann
84d0be50f0 update vulnerable moment package 2018-03-30 22:42:16 +02:00
Lukas F. Hartmann
6d2d2310b6 basic import functionality; dockerfile fixes; session and cookie handling fixes 2018-03-30 22:34:27 +02:00
mntmn
462e9edaab first importer WIP 2018-01-08 15:57:59 +01:00
mntmn
ffb7f30133 switch between s3/local storage in config 2018-01-08 12:43:47 +01:00
mntmn
8ee3386470 fix tmp dir in artifact uploader 2018-01-08 12:41:38 +01:00
mntmn
0d92343d55 fix redis mock incr() 2018-01-08 12:41:12 +01:00
mntmn
efb7970ecb fix express caching 2018-01-08 12:40:11 +01:00
mntmn
7e8a27e140 replace redis by in-memory object 2018-01-08 00:08:42 +01:00
mntmn
6ad97ac5c2 fix port number in readme 2018-01-07 22:54:31 +01:00
Lukas F. Hartmann
820203625c don't default to app volume mount; create s3 bucket on boot; fix revAll gulp step 2017-04-07 18:01:23 +02:00
Martin Guether
03059b67f1 add gulp stuff to build process 2017-04-07 15:34:33 +02:00
Martin Guether
1426bc9c24 updated storage config 2017-04-07 13:07:09 +02:00
Martin Guether
bd0471dad6 added docker ignore 2017-04-07 12:03:58 +02:00
Martin Guether
bbdcb2b2fe Merge remote-tracking branch 'origin/master' into docker
# Conflicts:
#	README.md
2017-04-07 12:00:28 +02:00
Martin Guether
af5335025f fixed crash on console 2017-04-07 11:58:29 +02:00
Martin Guether
f9cf8ba7e8 update ports and variables for docker compose bootstrap 2017-04-07 11:55:07 +02:00
Martin Guether
9877977750 add configuration lines to readme 2017-04-07 11:05:12 +02:00
Martin Guether
bda60bf877 add engine to package.json 2017-04-07 11:04:58 +02:00
Martin Guether
7605b8f373 node update 2017-04-07 10:59:10 +02:00
Martin Guether
dfcfd7f0b8 added binary dependencies to readme 2017-04-07 10:57:33 +02:00
Martin Guether
a3e2129b79 added experiemental docker support 2017-04-07 10:50:48 +02:00
Martin Guether
2e34b317a4 update packages 2017-04-07 10:39:35 +02:00
Martin Guether
d887d56dd3 fixed schema import for artifact 2017-04-07 10:39:24 +02:00
24 changed files with 491 additions and 122 deletions

14
.dockerignore Normal file
View File

@@ -0,0 +1,14 @@
.DS_Store
.git
logs
*.log
scripts
pids
*.pid
*.seed
lib-cov
coverage
.grunt
.lock-wscript
build/Release
node_modules

29
Dockerfile Normal file
View File

@@ -0,0 +1,29 @@
FROM spacedeck/docker-baseimage:latest
ENV NODE_ENV production
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
RUN npm install gulp-rev-replace gulp-clean gulp-fingerprint gulp-rev gulp-rev-all gulp-rev-replace
RUN npm install -g --save-dev gulp
COPY app.js Dockerfile Gulpfile.js LICENSE /usr/src/app/
COPY config /usr/src/app/config
COPY helpers /usr/src/app/helpers
COPY locales /usr/src/app/locales
COPY middlewares /usr/src/app/middlewares
COPY models /usr/src/app/models
COPY public /usr/src/app/public
COPY routes /usr/src/app/routes
COPY styles /usr/src/app/styles
COPY views /usr/src/app/views
RUN gulp all
RUN npm cache clean
CMD [ "node", "app.js" ]
EXPOSE 9666

View File

@@ -12,10 +12,9 @@ var uglify = require('gulp-uglify');
var fingerprint = require('gulp-fingerprint'); var fingerprint = require('gulp-fingerprint');
var rev = require('gulp-rev'); var rev = require('gulp-rev');
var RevAll = require('gulp-rev-all'); var revAll = require('gulp-rev-all');
gulp.task('rev', () => { gulp.task('rev', () => {
var revAll = new RevAll();
return gulp.src(['public/**']) return gulp.src(['public/**'])
.pipe(gulp.dest('build/assets')) .pipe(gulp.dest('build/assets'))
.pipe(revAll.revision()) .pipe(revAll.revision())

View File

@@ -1,7 +1,5 @@
# Spacedeck Open # Spacedeck Open
[![Greenkeeper badge](https://badges.greenkeeper.io/spacedeck/spacedeck-open.svg)](https://greenkeeper.io/)
This is the free and open source version of Spacedeck, a web based, real time, collaborative whiteboard application with rich media support. Spacedeck was developed in 6 major releases during Autumn 2011 until the end of 2016 and was originally a commercial SaaS. The developers were Lukas F. Hartmann (mntmn) and Martin Güther (magegu). All icons and large parts of the CSS were designed by Thomas Helbig (dergraph). This is the free and open source version of Spacedeck, a web based, real time, collaborative whiteboard application with rich media support. Spacedeck was developed in 6 major releases during Autumn 2011 until the end of 2016 and was originally a commercial SaaS. The developers were Lukas F. Hartmann (mntmn) and Martin Güther (magegu). All icons and large parts of the CSS were designed by Thomas Helbig (dergraph).
As we plan to retire the subscription based service at spacedeck.com in late 2017, we decided to open-source Spacedeck to allow educational and other organizations who currently rely on Spacedeck to migrate to a self-hosted version. As we plan to retire the subscription based service at spacedeck.com in late 2017, we decided to open-source Spacedeck to allow educational and other organizations who currently rely on Spacedeck to migrate to a self-hosted version.
@@ -25,16 +23,16 @@ We appreciate filed issues, pull requests and general discussion.
Spacedeck uses the following major building blocks: Spacedeck uses the following major building blocks:
- Node.js 4.x (Backend / API) - Vue.js (Frontend)
- Node.js 7.x (Backend / API)
- MongoDB 3.x (Datastore) - MongoDB 3.x (Datastore)
- Redis 3.x (Datastore for realtime channels) - Redis 3.x (Datastore for realtime channels)
- Vue.js (Frontend)
It also has some binary dependencies for media conversion and PDF export: It also has some binary dependencies for media conversion and PDF export:
- imagemagick - imagemagick, graphicsmagick, libav(+codecs, ffmpeg replacement), audiowaveform (https://github.com/bbcrd/audiowaveform), phantomjs (http://phantomjs.org/)
Currently, media files are stored in Amazon S3, so you need an Amazon AWS account and have the ```AWS_ACCESS_KEY_ID``` and ```AWS_SECRET_ACCESS_KEY``` environment variables defined. For sending emails, Amazon SES is required. Currently, media files are stored in Amazon S3, so you need an Amazon AWS account and have the ```AWS_ACCESS_KEY_ID``` and ```AWS_SECRET_ACCESS_KEY``` environment variables defined. For sending emails in production, Amazon SES is required.
To install Spacedeck, you need node.js 4.x and a running MongoDB instance. Then, to install all node dependencies, run To install Spacedeck, you need node.js 4.x and a running MongoDB instance. Then, to install all node dependencies, run
@@ -44,10 +42,22 @@ To rebuild the frontend CSS styles (you need to do this at least once):
gulp styles gulp styles
# Configuration
see: config/config.json
# Run # Run
export NODE_ENV=development export NODE_ENV=development
npm start npm start
open http://localhost:9666
# experimental docker(compose) support
We have a docker base image at https://github.com/spacedeck/docker-baseimage that includes all required binaries. Based on this image we can use Docker Compose to bootstrap a Spacedeck including data storage.
docker-compose build
docker-compose run -e ENV=development -p 9666:9666 -e NODE_ENV=development spacedeck-open
# License # License

19
app.js
View File

@@ -15,13 +15,16 @@ const favicon = require('serve-favicon');
const logger = require('morgan'); const logger = require('morgan');
const cookieParser = require('cookie-parser'); const cookieParser = require('cookie-parser');
const bodyParser = require('body-parser'); const bodyParser = require('body-parser');
const mongoose = require('mongoose'); const mongoose = require('mongoose');
const swig = require('swig'); const swig = require('swig');
const i18n = require('i18n-2'); const i18n = require('i18n-2');
const helmet = require('helmet'); const helmet = require('helmet');
const express = require('express'); const express = require('express');
const app = express(); const app = express();
const serveStatic = require('serve-static');
const isProduction = app.get('env') === 'production'; const isProduction = app.get('env') === 'production';
@@ -47,7 +50,7 @@ swig.setFilter('cdn', function(input, idx) {
app.engine('html', swig.renderFile); app.engine('html', swig.renderFile);
app.set('view engine', 'html'); app.set('view engine', 'html');
if (app.get('env') != 'development') { if (isProduction) {
app.set('views', path.join(__dirname, 'build', 'views')); app.set('views', path.join(__dirname, 'build', 'views'));
app.use(favicon(path.join(__dirname, 'build', 'assets', 'images', 'favicon.png'))); app.use(favicon(path.join(__dirname, 'build', 'assets', 'images', 'favicon.png')));
app.use(express.static(path.join(__dirname, 'build', 'assets'))); app.use(express.static(path.join(__dirname, 'build', 'assets')));
@@ -67,7 +70,6 @@ app.use(bodyParser.urlencoded({
})); }));
app.use(cookieParser()); app.use(cookieParser());
app.use(helmet.noCache())
app.use(helmet.frameguard()) app.use(helmet.frameguard())
app.use(helmet.xssFilter()) app.use(helmet.xssFilter())
app.use(helmet.hsts({ app.use(helmet.hsts({
@@ -82,7 +84,6 @@ app.use(helmet.noSniff())
app.use(require("./middlewares/templates")); app.use(require("./middlewares/templates"));
app.use(require("./middlewares/error_helpers")); app.use(require("./middlewares/error_helpers"));
app.use(require("./middlewares/setuser")); app.use(require("./middlewares/setuser"));
app.use(require("./middlewares/subdomain"));
app.use(require("./middlewares/cors")); app.use(require("./middlewares/cors"));
app.use(require("./middlewares/i18n")); app.use(require("./middlewares/i18n"));
app.use("/api", require("./middlewares/api_helpers")); app.use("/api", require("./middlewares/api_helpers"));
@@ -109,6 +110,12 @@ app.use('/api/teams', require('./routes/api/teams'));
app.use('/api/webgrabber', require('./routes/api/webgrabber')); app.use('/api/webgrabber', require('./routes/api/webgrabber'));
app.use('/', require('./routes/root')); app.use('/', require('./routes/root'));
if (config.get('storage_local_path')) {
app.use('/storage', serveStatic(config.get('storage_local_path')+"/"+config.get('storage_bucket'), {
maxAge: 24*3600
}));
}
// catch 404 and forward to error handler // catch 404 and forward to error handler
app.use(require('./middlewares/404')); app.use(require('./middlewares/404'));
if (app.get('env') == 'development') { if (app.get('env') == 'development') {
@@ -121,14 +128,14 @@ if (app.get('env') == 'development') {
module.exports = app; module.exports = app;
// CONNECT TO DATABASE // CONNECT TO DATABASE
const mongoHost = process.env.MONGO_PORT_27017_TCP_ADDR || 'localhost'; const mongoHost = process.env.MONGO_PORT_27017_TCP_ADDR || config.get('mongodb_host');
mongoose.connect('mongodb://' + mongoHost + '/spacedeck'); mongoose.connect('mongodb://' + mongoHost + '/spacedeck');
// START WEBSERVER // START WEBSERVER
const port = 9000; const port = 9666;
const server = http.Server(app).listen(port, () => { const server = http.Server(app).listen(port, () => {
if ("send" in process) { if ("send" in process) {
process.send('online'); process.send('online');
} }

View File

@@ -1,7 +1,20 @@
{ {
"endpoint": "http://localhost:9000", //"endpoint": "http://localhost:9000",
"storage_bucket": "my_spacedeck_s3_bucket", "endpoint": "http://localhost:9666",
"storage_cdn": "xyz.cloudfront.net", "storage_region": "eu-central-1",
//"storage_bucket": "sdeck-development",
//"storage_cdn": "http://localhost:9123/sdeck-development",
//"storage_endpoint": "http://storage:9000",
"storage_bucket": "my_spacedeck_bucket",
"storage_cdn": "/storage",
"storage_local_path": "./storage",
"redis_mock": true,
"mongodb_host": "localhost",
"redis_host": "localhost",
"google_access" : "", "google_access" : "",
"google_secret" : "", "google_secret" : "",
"admin_pass": "very_secret_admin_password", "admin_pass": "very_secret_admin_password",

34
docker-compose.yml Normal file
View File

@@ -0,0 +1,34 @@
version: '2'
services:
sync:
image: redis
storage:
image: minio/minio
environment:
- MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
- MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ports:
- 9123:9000
command: server /export
db:
image: mongo
spacedeck-open:
environment:
- env=development
- MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
- MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
build: .
volumes:
# - ./:/usr/src/app
- /usr/src/app/node_modules
command: npm start
ports:
- 9666:9666
depends_on:
- db
- sync
- storage
links:
- storage
- db
- sync

View File

@@ -7,6 +7,7 @@ const fs = require('fs');
const Models = require('../models/schema'); const Models = require('../models/schema');
const uploader = require('../helpers/uploader'); const uploader = require('../helpers/uploader');
const path = require('path'); const path = require('path');
const os = require('os');
const fileExtensionMap = { const fileExtensionMap = {
".amr" : "audio/AMR", ".amr" : "audio/AMR",
@@ -245,13 +246,12 @@ function resizeAndUpload(a, size, max, fileName, localFilePath, callback) {
if (max>320 || size.width > max || size.height > max) { if (max>320 || size.width > max || size.height > max) {
var resizedFileName = max + "_"+fileName; var resizedFileName = max + "_"+fileName;
var s3Key = "s"+ a.space_id.toString() + "/a" + a._id.toString() + "/" + resizedFileName; var s3Key = "s"+ a.space_id.toString() + "/a" + a._id.toString() + "/" + resizedFileName;
var localResizedFilePath = "/tmp/"+resizedFileName; var localResizedFilePath = os.tmpdir()+"/"+resizedFileName;
gm(localFilePath).resize(max, max).autoOrient().write(localResizedFilePath, function (err) { gm(localFilePath).resize(max, max).autoOrient().write(localResizedFilePath, function (err) {
if(!err) { if(!err) {
uploader.uploadFile(s3Key, "image/jpeg", localResizedFilePath, function(err, url) { uploader.uploadFile(s3Key, "image/jpeg", localResizedFilePath, function(err, url) {
if (err) callback(err); if (err) callback(err);
else{ else{
console.log(localResizedFilePath);
fs.unlink(localResizedFilePath, function (err) { fs.unlink(localResizedFilePath, function (err) {
if (err) { if (err) {
console.error(err); console.error(err);

103
helpers/importer.js Normal file
View File

@@ -0,0 +1,103 @@
'use strict';
const extract = require('extract-zip')
const config = require('config')
const fs = require('fs')
const path = require('path')
require('../models/schema')
module.exports = {
importZIP: function(user, zipPath) {
// 1. extract zip to local storage folder
// 2. read spaces.json from this folder
// 3. iterate through spaces and read all their artifact jsons
// 4. fixup storage paths
// 5. replace creator id by user._id
let relativeImportDir = 'import_'+user._id
let importDir = path.resolve(config.get('storage_local_path')+'/'+config.get('storage_bucket')+'/'+relativeImportDir)
if (!fs.existsSync(importDir)) {
fs.mkdirSync(importDir)
}
extract(zipPath, {dir: importDir}, function(err) {
if (err) {
console.log(err)
return
}
console.log('[import] extracted to',importDir)
let spacesJson = fs.readFileSync(importDir+'/spaces.json')
let spaces = JSON.parse(spacesJson)
var homeFolderId = null
console.log('[import] spaces:',spaces.length)
// pass 1: find homefolder
for (var i=0; i<spaces.length; i++) {
let space = spaces[i]
if (!space.parent_space_id) {
homeFolderId = space._id
break
}
}
console.log("[import] homeFolderId:",homeFolderId)
for (var i=0; i<spaces.length; i++) {
let space = spaces[i]
if (space.parent_space_id) {
let artifacts = JSON.parse(fs.readFileSync(importDir+'/'+space._id+'_artifacts.json'))
console.log('[import] space',space._id,'artifacts:',artifacts.length)
let q = {_id: space._id}
space.creator = user._id
delete space.__v
// transplant homefolder
console.log("parent:",space.parent_space_id)
if (space.parent_space_id+"" == homeFolderId+"") {
space.parent_space_id = user.home_folder_id
}
Space.findOneAndUpdate(q, space, {upsert: true}, function(err,res) {
if (err) console.log("[import] space upsert err:",err)
})
for (var j=0; j<artifacts.length; j++) {
let a = artifacts[j]
let q = {_id: a._id}
a.creator = user._id
delete a.__v
delete a.payload_thumbnail_big_uri
let prefix = "/storage/"+relativeImportDir+"/"+space._id+"_files/"
if (a.thumbnail_uri && a.thumbnail_uri[0]!='/') a.thumbnail_uri = prefix + a.thumbnail_uri
if (a.payload_uri && a.payload_uri[0]!='/') a.payload_uri = prefix + a.payload_uri
if (a.payload_thumbnail_web_uri && a.payload_thumbnail_web_uri[0]!='/') a.payload_thumbnail_web_uri = prefix + a.payload_thumbnail_web_uri
if (a.payload_thumbnail_medium_uri && a.payload_thumbnail_medium_uri[0]!='/') a.payload_thumbnail_medium_uri = prefix + a.payload_thumbnail_medium_uri
if (a.payload_alternatives) {
for (var k=0; k<a.payload_alternatives.length; k++) {
let alt = a.payload_alternatives[k]
if (alt.payload_uri && alt.payload_uri[0]!='/') alt.payload_uri = prefix + alt.payload_uri
if (alt.payload_thumbnail_web_uri && alt.payload_thumbnail_web_uri[0]!='/') alt.payload_thumbnail_web_uri = prefix + alt.payload_thumbnail_web_uri
if (alt.payload_thumbnail_medium_uri && alt.payload_thumbnail_medium_uri[0]!='/') alt.payload_thumbnail_medium_uri = prefix + alt.payload_thumbnail_medium_uri
}
}
Artifact.findOneAndUpdate(q, a, {upsert: true}, function(err,res) {
if (err) console.log("[import] artifact upsert err:",err)
})
}
}
}
})
}
}

View File

@@ -53,7 +53,7 @@ module.exports = {
} }
} }
}, function(err, data) { }, function(err, data) {
if(err) console.log('Email not sent:', err); if (err) console.error("Error sending email:", err);
else console.log("Email sent."); else console.log("Email sent.");
}); });
} }

View File

@@ -32,31 +32,36 @@ module.exports = {
}; };
phantom.create({ path: require('phantomjs-prebuilt').path }, function (err, browser) { phantom.create({ path: require('phantomjs-prebuilt').path }, function (err, browser) {
return browser.createPage(function (err, page) { if(err){
console.log("page created, opening ",space_url); console.log(err);
}else{
return browser.createPage(function (err, page) {
console.log("page created, opening ",space_url);
if (type=="pdf") { if (type=="pdf") {
var psz = { var psz = {
width: space.advanced.width+"px", width: space.advanced.width+"px",
height: space.advanced.height+"px" height: space.advanced.height+"px"
}; };
page.set('paperSize', psz); page.set('paperSize', psz);
} }
page.set('settings.resourceTimeout',timeout); page.set('settings.resourceTimeout',timeout);
page.set('settings.javascriptEnabled',false); page.set('settings.javascriptEnabled',false);
return page.open(space_url, function (err,status) { return page.open(space_url, function (err,status) {
page.render(export_path, function() { page.render(export_path, function() {
on_success_called = true; on_success_called = true;
if (on_success) { if (on_success) {
on_success(export_path); on_success(export_path);
} }
page.close(); page.close();
browser.exit(); browser.exit();
});
}); });
}); });
}); }
}, { }, {
onExit: on_exit onExit: on_exit
}); });

View File

@@ -1,14 +1,111 @@
'use strict'; 'use strict';
const RedisConnection = require('ioredis'); const config = require('config');
const websockets = require('./websockets');
// this is a mock version of the Redis API,
// emulating Redis if it is not available locally
var notRedis = {
state: {},
topics: {},
publish: function(topic, msg, cb) {
if (!this.topics[topic]) {
this.topics[topic] = {
subscribers: []
};
}
var t=this.topics[topic];
for (var i=0; i<t.subscribers.length; i++) {
var s=t.subscribers[i];
if (s.handler) {
s.handler(topic, msg);
}
}
if (cb) cb(null);
},
subscribe: function(topics, cb) {
var handle = {
handler: null,
on: function(evt, cb) {
if (evt == "message") {
this.handler = cb;
}
}
};
for (var i=0; i<topics.length; i++) {
var topic = topics[i];
if (!this.topics[topic]) {
this.topics[topic] = {
subscribers: []
};
}
var t=this.topics[topic];
t.subscribers.push(handle);
}
cb(null, topics.length);
return handle;
},
get: function(key, cb) {
cb(null, this.state[key]);
return this.state[key];
},
set: function(key, val, cb) {
this.state[key] = val;
cb();
},
del: function(key, cb) {
delete this.state[key];
cb(null);
},
sadd: function(key, skey, cb) {
if (!this.state[key]) this.state[key] = {};
this.state[key][skey] = true;
cb(null);
},
srem: function(key, skey, cb) {
if (this.state[key]) {
delete this.state[key][skey];
}
cb(null);
},
smembers: function(key, cb) {
cb(null, Object.keys(this.state[key]));
},
incr: function(key, cb) {
if (!this.state[key]) this.state[key] = 0;
this.state[key]++;
cb(null, this.state[key]);
},
expire: function() {
},
}
module.exports = { module.exports = {
connectRedis(){ connectRedis: function() {
const redisHost = process.env.REDIS_PORT_6379_TCP_ADDR || 'localhost'; if (config.get("redis_mock")) {
this.connection = new RedisConnection(6379, redisHost); this.connection = notRedis;
} else {
const redisHost = process.env.REDIS_PORT_6379_TCP_ADDR || 'sync';
this.connection = new RedisConnection(6379, redisHost);
}
}, },
sendMessage(action, model, attributes, channelId) { getConnection: function() {
this.connectRedis();
return this.connection;
},
sendMessage: function(action, model, attributes, channelId) {
const data = JSON.stringify({ const data = JSON.stringify({
channel_id: channelId, channel_id: channelId,
action: action, action: action,
@@ -17,12 +114,12 @@ module.exports = {
}); });
this.connection.publish('updates', data); this.connection.publish('updates', data);
}, },
logIp(ip, cb) { logIp: function(ip, cb) {
this.connection.incr("ip_"+ ip, (err, socketCounter) => { this.connection.incr("ip_"+ ip, (err, socketCounter) => {
cb(); cb();
}); });
}, },
rateLimit(namespace, ip, cb) { rateLimit: function(namespace, ip, cb) {
const key = "limit_"+ namespace + "_"+ ip; const key = "limit_"+ namespace + "_"+ ip;
const redis = this.connection; const redis = this.connection;
@@ -47,7 +144,7 @@ module.exports = {
} }
}); });
}, },
isOnlineInSpace(user, space, cb) { isOnlineInSpace: function(user, space, cb) {
this.connection.smembers("space_" + space._id.toString(), function(err, list) { this.connection.smembers("space_" + space._id.toString(), function(err, list) {
if (err) cb(err); if (err) cb(err);
else { else {
@@ -59,3 +156,6 @@ module.exports = {
}); });
} }
}; };
return module.exports;

View File

@@ -1,16 +1,41 @@
'use strict'; 'use strict';
var AWS = require('aws-sdk');
AWS.config.region = 'eu-central-1';
var fs = require('fs'); var fs = require('fs');
var config = require('config'); var config = require('config');
var s3 = null;
// use AWS S3 or local folder depending on config
if (config.get("storage_local_path")) {
var AWS = require('mock-aws-s3');
AWS.config.basePath = config.get("storage_local_path");
s3 = new AWS.S3();
} else {
var AWS = require('aws-sdk');
var storage_endpoint = config.get("storage_endpoint");
const ep = new AWS.Endpoint(storage_endpoint);
AWS.config.update(new AWS.Config({
accessKeyId: process.env.MINIO_ACCESS_KEY,
secretAccessKey: process.env.MINIO_SECRET_KEY,
region: config.get("storage_region"),
s3ForcePathStyle: true,
signatureVersion: 'v4'
}));
s3 = new AWS.S3({
endpoint: ep
});
}
s3.createBucket({
Bucket: config.get("storage_bucket"),
ACL: "public-read",
GrantRead: "*"
}, (err,res) => {
console.log("createBucket",err,res);
});
module.exports = { module.exports = {
removeFile: (path, callback) => { removeFile: (path, callback) => {
const s3 = new AWS.S3({
region: 'eu-central-1'
});
const bucket = config.get("storage_bucket"); const bucket = config.get("storage_bucket");
s3.deleteObject({ s3.deleteObject({
Bucket: bucket, Key: path Bucket: bucket, Key: path
@@ -28,7 +53,7 @@ module.exports = {
callback({error:"missing path"}, null); callback({error:"missing path"}, null);
return; return;
} }
console.log("[s3] uploading", localFilePath, " to ", fileName); console.log("[storage] uploading", localFilePath, " to ", fileName);
const bucket = config.get("storage_bucket"); const bucket = config.get("storage_bucket");
const fileStream = fs.createReadStream(localFilePath); const fileStream = fs.createReadStream(localFilePath);
@@ -39,11 +64,6 @@ module.exports = {
} }
}); });
fileStream.on('open', function () { fileStream.on('open', function () {
// FIXME
var s3 = new AWS.S3({
region: 'eu-central-1'
});
s3.putObject({ s3.putObject({
Bucket: bucket, Bucket: bucket,
Key: fileName, Key: fileName,
@@ -53,8 +73,8 @@ module.exports = {
if (err){ if (err){
console.error(err); console.error(err);
callback(err); callback(err);
}else { } else {
const url = "https://"+ config.get("storage_cdn") + "/" + fileName; const url = config.get("storage_cdn") + "/" + fileName;
console.log("[s3]" + localFilePath + " to " + url); console.log("[s3]" + localFilePath + " to " + url);
callback(null, url); callback(null, url);
} }

View File

@@ -1,20 +1,28 @@
'use strict'; 'use strict';
require('../models/schema'); require('../models/schema');
const config = require('config');
const WebSocketServer = require('ws').Server; const WebSocketServer = require('ws').Server;
const Redis = require('ioredis'); const RedisConnection = require('ioredis');
const async = require('async'); const async = require('async');
const _ = require("underscore"); const _ = require("underscore");
const mongoose = require("mongoose"); const mongoose = require("mongoose");
const crypto = require('crypto'); const crypto = require('crypto');
module.exports = { const redisMock = require("./redis.js");
startWebsockets: function(server){
this.setupSubscription();
this.state = new Redis(6379, process.env.REDIS_PORT_6379_TCP_ADDR || 'localhost');
if(!this.current_websockets){ module.exports = {
startWebsockets: function(server) {
this.setupSubscription();
if (!this.current_websockets) {
if (config.get("redis_mock")) {
this.state = redisMock.getConnection();
} else {
this.state = new RedisConnection(6379, process.env.REDIS_PORT_6379_TCP_ADDR || config.get("redis_host"));
}
this.current_websockets = []; this.current_websockets = [];
} }
@@ -117,10 +125,17 @@ module.exports = {
}, },
setupSubscription: function() { setupSubscription: function() {
this.cursorSubscriber = new Redis(6379, process.env.REDIS_PORT_6379_TCP_ADDR || 'localhost'); if (config.get("redis_mock")) {
this.cursorSubscriber.subscribe(['cursors', 'users', 'updates'], function (err, count) { this.cursorSubscriber = redisMock.getConnection().subscribe(['cursors', 'users', 'updates'], function (err, count) {
console.log("[redis] websockets to " + count + " topics." ); console.log("[redis-mock] websockets subscribed to " + count + " topics." );
}); });
} else {
this.cursorSubscriber = new RedisConnection(6379, process.env.REDIS_PORT_6379_TCP_ADDR || config.get("redis_host"));
this.cursorSubscriber.subscribe(['cursors', 'users', 'updates'], function (err, count) {
console.log("[redis] websockets subscribed to " + count + " topics." );
});
}
this.cursorSubscriber.on('message', function (channel, rawMessage) { this.cursorSubscriber.on('message', function (channel, rawMessage) {
const msg = JSON.parse(rawMessage); const msg = JSON.parse(rawMessage);
const spaceId = msg.space_id; const spaceId = msg.space_id;
@@ -206,7 +221,7 @@ module.exports = {
console.log("websocket not found to remove"); console.log("websocket not found to remove");
} }
this.state.del(ws.id, function(err, res) { this.state.del(ws.id+"", function(err, res) {
if (err) console.error(err, res); if (err) console.error(err, res);
else { else {
this.removeUserInSpace(ws.space_id, ws, (err) => { this.removeUserInSpace(ws.space_id, ws, (err) => {
@@ -221,7 +236,8 @@ module.exports = {
addUserInSpace: function(username, space, ws, cb) { addUserInSpace: function(username, space, ws, cb) {
console.log("[websockets] user "+username+" in "+space.access_mode +" space " + space._id + " with socket " + ws.id); console.log("[websockets] user "+username+" in "+space.access_mode +" space " + space._id + " with socket " + ws.id);
this.state.set(ws.id, username, function(err, res) {
this.state.set(ws.id+"", username+"", function(err, res) {
if(err) console.error(err, res); if(err) console.error(err, res);
else { else {
this.state.sadd("space_" + space._id, ws.id, function(err, res) { this.state.sadd("space_" + space._id, ws.id, function(err, res) {
@@ -238,7 +254,7 @@ module.exports = {
}.bind(this)); }.bind(this));
}, },
removeUserInSpace: function(spaceId, ws, cb) { removeUserInSpace: function(spaceId, ws, cb) {
this.state.srem("space_" + spaceId, ws.id, function(err, res) { this.state.srem("space_" + spaceId, ws.id+"", function(err, res) {
if (err) cb(err); if (err) cb(err);
else { else {
console.log("[websockets] socket "+ ws.id + " went offline in space " + spaceId); console.log("[websockets] socket "+ ws.id + " went offline in space " + spaceId);

View File

@@ -5,27 +5,24 @@ var config = require('config');
module.exports = (req, res, next) => { module.exports = (req, res, next) => {
const token = req.cookies["sdsession"]; const token = req.cookies["sdsession"];
if (token && token != "null" && token !== null) { if (token && token != "null" && token !== null) {
User.findOne({ User.findOne({
"sessions.token": token "sessions.token": token
}).populate('team').exec((err, user) => { }).populate('team').exec((err, user) => {
if (err) console.error("session.token lookup error:",err);
if (!user) { if (!user) {
// FIXME res.clearCookie('sdsession');
var domain = "localhost";
res.clearCookie('sdsession', {
domain: domain
});
if (req.accepts("text/html")) { if (req.accepts("text/html")) {
res.redirect("/"); res.send("Please clear your cookies and try again.");
} else if (req.accepts('application/json')) { } else if (req.accepts('application/json')) {
res.status(403).json({ res.status(403).json({
"error": "token_not_found" "error": "token_not_found"
}); });
} else { } else {
res.redirect("/"); res.send("Please clear your cookies and try again.");
} }
} else { } else {
req["token"] = token; req["token"] = token;
req["user"] = user; req["user"] = user;

View File

@@ -3,7 +3,7 @@
var mongoose = require('mongoose'); var mongoose = require('mongoose');
var Schema = mongoose.Schema; var Schema = mongoose.Schema;
module.exports.artifactSchema = mongoose.model('Artifact', { module.exports.artifactSchema = Schema({
mime: String, mime: String,
thumbnail_uri: String, thumbnail_uri: String,
space_id: Schema.Types.ObjectId, space_id: Schema.Types.ObjectId,

View File

@@ -45,7 +45,7 @@ module.exports.teamSchema.index({
module.exports.teamSchema.statics.getTeamForHost = (host, cb) => { module.exports.teamSchema.statics.getTeamForHost = (host, cb) => {
if (host != "127.0.0.1:9000") { //phantomjs check if (host != "127.0.0.1:9666") { //phantomjs check
let subDomainParts = host.split('.'); let subDomainParts = host.split('.');
if (subDomainParts.length > 2) { if (subDomainParts.length > 2) {

View File

@@ -6,12 +6,15 @@
"start": "nodemon -e .js,.html bin/www", "start": "nodemon -e .js,.html bin/www",
"test": "mocha" "test": "mocha"
}, },
"engines": {
"node": ">=7.8.0"
},
"dependencies": { "dependencies": {
"archiver": "1.3.0", "archiver": "1.3.0",
"async": "2.3.0", "async": "2.3.0",
"aws-sdk": "2.39.0", "aws-sdk": "2.39.0",
"basic-auth": "1.1.0", "basic-auth": "1.1.0",
"bcrypt": "*", "bcryptjs": "2.4.3",
"body-parser": "~1.17.1", "body-parser": "~1.17.1",
"cheerio": "0.22.0", "cheerio": "0.22.0",
"config": "1.25.1", "config": "1.25.1",
@@ -19,17 +22,17 @@
"csurf": "1.9.0", "csurf": "1.9.0",
"debug": "~2.6.3", "debug": "~2.6.3",
"execSync": "latest", "execSync": "latest",
"express": "~4.15.2", "express": "~4.13.0",
"extract-zip": "^1.6.6",
"glob": "7.1.1", "glob": "7.1.1",
"gm": "1.23.0", "gm": "1.23.0",
"googleapis": "18.0.0", "googleapis": "18.0.0",
"gulp": "^3.9.0", "gulp": "^3.9.1",
"gulp-concat": "2.6.1", "gulp-concat": "2.6.0",
"gulp-eslint": "*", "gulp-express": "0.3.0",
"gulp-express": "0.3.5",
"gulp-nodemon": "*", "gulp-nodemon": "*",
"gulp-sass": "^3.1.0", "gulp-sass": "^2.0.3",
"gulp-uglify": "^2.1.2", "gulp-uglify": "^1.5.1",
"gulp-util": "^3.0.6", "gulp-util": "^3.0.6",
"helmet": "^3.5.0", "helmet": "^3.5.0",
"i18n-2": "0.6.3", "i18n-2": "0.6.3",
@@ -37,13 +40,13 @@
"lodash": "^4.3.0", "lodash": "^4.3.0",
"log-timestamp": "latest", "log-timestamp": "latest",
"md5": "2.2.1", "md5": "2.2.1",
"moment": "2.18.1", "mock-aws-s3": "^2.6.0",
"moment": "^2.19.3",
"mongoose": "4.9.3", "mongoose": "4.9.3",
"morgan": "1.8.1", "morgan": "1.8.1",
"node-phantom-simple": "2.2.4",
"node-sass-middleware": "0.11.0", "node-sass-middleware": "0.11.0",
"pdfkit": "0.8.0", "pdfkit": "0.8.0",
"validator": "7.0.0",
"node-phantom-simple": "2.2.4",
"phantomjs-prebuilt": "2.1.14", "phantomjs-prebuilt": "2.1.14",
"pm2": "latest", "pm2": "latest",
"qr-image": "3.2.0", "qr-image": "3.2.0",
@@ -51,18 +54,19 @@
"request": "2.81.0", "request": "2.81.0",
"sanitize-html": "^1.11.1", "sanitize-html": "^1.11.1",
"serve-favicon": "~2.4.2", "serve-favicon": "~2.4.2",
"swig": "1.4.2", "serve-static": "^1.13.1",
"slug": "0.9.1", "slug": "0.9.1",
"swig": "1.4.2",
"underscore": "1.8.3", "underscore": "1.8.3",
"validator": "7.0.0",
"weak": "1.0.1", "weak": "1.0.1",
"ws": "2.2.3" "ws": "2.2.3"
}, },
"devDependencies": { "devDependencies": {
"express": "^4.13.3", "express": "^4.13.3",
"gulp": "^3.9.0", "gulp": "^3.9.1",
"gulp-clean": "^0.3.2", "gulp-clean": "^0.3.2",
"gulp-concat": "^2.6.0", "gulp-concat": "^2.6.0",
"gulp-eslint": "^3.0.1",
"gulp-express": "^0.3.0", "gulp-express": "^0.3.0",
"gulp-fingerprint": "^0.3.2", "gulp-fingerprint": "^0.3.2",
"gulp-nodemon": "^2.0.4", "gulp-nodemon": "^2.0.4",
@@ -71,16 +75,14 @@
"gulp-rev-replace": "^0.4.3", "gulp-rev-replace": "^0.4.3",
"gulp-sass": "^3.1.0", "gulp-sass": "^3.1.0",
"gulp-uglify": "^2.1.2", "gulp-uglify": "^2.1.2",
"mocha": "*", "nodemon": "1.11.0",
"nodemon": "*",
"should": "^11.2.1", "should": "^11.2.1",
"supertest": "^3.0.0", "supertest": "^3.0.0",
"winston": "^2.3.1" "winston": "^2.3.1"
}, },
"description": "", "description": "",
"main": "Gulpfile.js", "main": "Gulpfile.js",
"directories": { "directories": {},
},
"repository": { "repository": {
"type": "git", "type": "git",
"url": "https://github.com/spacedeck/spacedeck-open.git" "url": "https://github.com/spacedeck/spacedeck-open.git"

View File

@@ -3,8 +3,9 @@
var config = require('config'); var config = require('config');
require('../../models/schema'); require('../../models/schema');
var bcrypt = require('bcrypt'); var bcrypt = require('bcryptjs');
var crypo = require('crypto'); var crypo = require('crypto');
var URL = require('url').URL;
var express = require('express'); var express = require('express');
var router = express.Router(); var router = express.Router();
@@ -40,11 +41,11 @@ router.post('/', function(req, res) {
user.sessions.push(session); user.sessions.push(session);
user.save(function(err, result) { user.save(function(err, result) {
// FIXME if (err) console.error("Error saving user:",err);
var secure = process.env.NODE_ENV == "production" || process.env.NODE_ENV == "staging";
var domain = (process.env.NODE_ENV == "production") ? ".example.org" : "localhost"; var domain = (process.env.NODE_ENV == "production") ? new URL(config.get('endpoint')).hostname : "localhost";
res.cookie('sdsession', token, { domain: domain, httpOnly: true, secure: secure}); res.cookie('sdsession', token, { domain: domain, httpOnly: true });
res.status(201).json(session); res.status(201).json(session);
}); });
}); });
@@ -69,8 +70,7 @@ router.delete('/current', function(req, res, next) {
}); });
user.sessions = newSessions; user.sessions = newSessions;
user.save(function(err, result) { user.save(function(err, result) {
// FIXME var domain = new URL(config.get('endpoint')).hostname;
var domain = (process.env.NODE_ENV == "production") ? ".example.org" : "localhost";
res.clearCookie('sdsession', { domain: domain }); res.clearCookie('sdsession', { domain: domain });
res.sendStatus(204); res.sendStatus(204);
}); });

View File

@@ -59,7 +59,9 @@ router.get('/', (req, res) => {
"nickname": 1, "nickname": 1,
"email": 1 "email": 1
}).exec((err, user) => { }).exec((err, user) => {
a['user'] = user.toObject(); if (user) {
a['user'] = user.toObject();
}
cb(err, a); cb(err, a);
}); });
} else { } else {

View File

@@ -9,7 +9,7 @@ var mailer = require('../../helpers/mailer');
var fs = require('fs'); var fs = require('fs');
var _ = require('underscore'); var _ = require('underscore');
var crypto = require('crypto'); var crypto = require('crypto');
var bcrypt = require('bcrypt'); var bcrypt = require('bcryptjs');
var express = require('express'); var express = require('express');
var router = express.Router(); var router = express.Router();

View File

@@ -5,8 +5,9 @@ require('../../models/schema');
var mailer = require('../../helpers/mailer'); var mailer = require('../../helpers/mailer');
var uploader = require('../../helpers/uploader'); var uploader = require('../../helpers/uploader');
var importer = require('../../helpers/importer');
var bcrypt = require('bcrypt'); var bcrypt = require('bcryptjs');
var crypo = require('crypto'); var crypo = require('crypto');
var swig = require('swig'); var swig = require('swig');
var async = require('async'); var async = require('async');
@@ -15,6 +16,7 @@ var fs = require('fs');
var request = require('request'); var request = require('request');
var gm = require('gm'); var gm = require('gm');
var validator = require('validator'); var validator = require('validator');
var URL = require('url').URL;
var express = require('express'); var express = require('express');
var router = express.Router(); var router = express.Router();
@@ -181,8 +183,7 @@ router.get('/loginorsignupviagoogle', function(req, res) {
var apiUrl = "https://www.googleapis.com/oauth2/v1/userinfo?alt=json&access_token=" + tokens.access_token; var apiUrl = "https://www.googleapis.com/oauth2/v1/userinfo?alt=json&access_token=" + tokens.access_token;
var finalizeLogin = function(session){ var finalizeLogin = function(session){
var secure = process.env.NODE_ENV == "production" || process.env.NODE_ENV == "staging"; res.cookie('sdsession', session.token, { httpOnly: true });
res.cookie('sdsession', session.token, { httpOnly: true, secure: secure});
res.status(201).json(session); res.status(201).json(session);
}; };
@@ -467,4 +468,13 @@ router.post('/:user_id/confirm', function(req, res, next) {
res.sendStatus(201); res.sendStatus(201);
}); });
router.get('/:user_id/import', function(req, res, next) {
if (req.query.zip) {
res.send("importing");
importer.importZIP(req.user, req.query.zip);
} else {
res.sendStatus(400);
}
});
module.exports = router; module.exports = router;

View File

@@ -53,7 +53,7 @@
<p> <p>
<div class="col-xs-6"> <div class="col-xs-6">
<a href="/contact">[[ __("contact") ]]</a> <a href="/contact">[[ __("contact") ]]</a>
<span style="color:#888">&copy; 20112017 The Spacedeck Open Developers</span> <span style="color:#888">&copy; 20112018 The Spacedeck Open Developers <a href="https://github.com/spacedeck/spacedeck-open">https://github.com/spacedeck/spacedeck-open</a></span>
</div> </div>
</p> </p>
</div> </div>

View File

@@ -23,10 +23,18 @@
{% if process.env.NODE_ENV != "production" %} {% if process.env.NODE_ENV != "production" %}
var ENV = { var ENV = {
name: 'development', name: 'development',
webHost: "localhost:9000", webHost: "localhost:9666",
webEndpoint:"http://localhost:9000", webEndpoint:"http://localhost:9666",
apiEndpoint: "http://localhost:9000", apiEndpoint: "http://localhost:9666",
websocketsEndpoint: "ws://localhost:9000" websocketsEndpoint: "ws://localhost:9666"
};
{% else %}
var ENV = {
name: 'production',
webHost: location.host,
webEndpoint: location.origin,
apiEndpoint: location.origin,
websocketsEndpoint: location.origin.replace("https:","wss:").replace("http:","ws:")
}; };
{% endif %} {% endif %}