[MediaWiki-commits] [Gerrit] Add 'phaste' tool for pastebinning text to Phabricator - change (operations/puppet)

[Ori.livneh (Code Review)]

Ori.livneh has uploaded a new change for review.

https://gerrit.wikimedia.org/r/174297

Change subject: Add 'phaste' tool for pastebinning text to Phabricator
......................................................................

Add 'phaste' tool for pastebinning text to Phabricator

Provides the ability to pastebin text on Wikimedia's Phabricator installation
from anywhere in production by running 'phaste' -- e.g.:

# read from stdin:
grep xfs /var/log/syslog | phaste

# read from file:
phaste /var/log/upstart/service.log

Change-Id: I4705178445e4f1437cb606f92f2108755c095e87
---
M modules/base/manifests/init.pp
A modules/phaste/files/phaste
A modules/phaste/manifests/init.pp
3 files changed, 128 insertions(+), 0 deletions(-)


git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/97/174297/1

diff --git a/modules/base/manifests/init.pp b/modules/base/manifests/init.pp
index ab83082..21b232b 100644
--- a/modules/base/manifests/init.pp
+++ b/modules/base/manifests/init.pp
@@ -255,4 +255,13 @@
# TODO: Kill the old wmf_ca
include certificates::wmf_ca
include certificates::wmf_ca_2014_2017
+
+
+ include passwords::phabricator
+
+ class { 'phaste':
+ user => 'ProdPasteBot',
+ cert => $passwords::phabricator::pastebot_cert,
+ phab => 'https://phabricator.wikimedia.org',
+ }
}
diff --git a/modules/phaste/files/phaste b/modules/phaste/files/phaste
new file mode 100755
index 0000000..3d1f78b
--- /dev/null
+++ b/modules/phaste/files/phaste
@@ -0,0 +1,70 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+ phaste: Phabricator paste tool
+
+"""
+import sys
+reload(sys)
+sys.setdefaultencoding('utf-8')
+
+import fileinput
+import hashlib
+import json
+import time
+import urllib
+import urllib2
+
+
+class Conduit(object):
+ """A wrapper around Phabricator's Conduit API."""
+
+ def __init__(self, phab, user, cert):
+ self.phab = phab
+ self.user = user
+ self.cert = cert
+ self.credentials = None
+
+ def _do_request(self, method, data):
+ url = '%s/api/%s' % (self.phab, method)
+ req = urllib2.urlopen(url, data=urllib.urlencode(data))
+ resp = json.load(req)
+ if resp.get('error_info'):
+ raise RuntimeError('%(error_code)s: %(error_info)s' % resp)
+ return resp['result']
+
+ def _get_credentials(self):
+ if self.credentials is None:
+ token = int(time.time())
+ signature = hashlib.sha1(str(token) + self.cert).hexdigest()
+ params = {
+ 'client': 'phaste',
+ 'clientVersion': 0,
+ 'user': self.user,
+ 'authToken': token,
+ 'authSignature': signature,
+ }
+ data = {
+ 'params': json.dumps(params),
+ 'output': 'json',
+ '__conduit__': 'true',
+ }
+ self.credentials = self._do_request('conduit.connect', data)
+ return self.credentials
+
+ def call(self, method, **params):
+ """Call the conduit API."""
+ params['__conduit__'] = self._get_credentials()
+ data = {
+ 'params': json.dumps(params),
+ 'output': 'json',
+ }
+ return self._do_request(method, data)
+
+
+with open('/etc/phaste.conf') as f:
+ config = json.load(f)
+
+p = Conduit(phab=config['phab'], user=config['user'], cert=config['cert'])
+res = p.call('paste.create', content=''.join(fileinput.input()))
+print res['uri']
diff --git a/modules/phaste/manifests/init.pp b/modules/phaste/manifests/init.pp
new file mode 100644
index 0000000..2b47033
--- /dev/null
+++ b/modules/phaste/manifests/init.pp
@@ -0,0 +1,49 @@
+# == Class: phaste
+#
+# Description
+#
+# === Parameters
+#
+# [*user*]
+# Username the bot should use to authenticate.
+#
+# [*cert*]
+# Conduit certificate for the phaste user.
+#
+# [*phab*]
+# URL of Phabricator instance.
+#
+# === Examples
+#
+# class { 'phaste':
+# user => 'ProdPasteBot',
+# cert => 'dmalem5s...',
+# phab => 'https://phabricator.wikimedia.org',
+# }
+#
+class phaste {
+ $user,
+ $cert,
+ $phab,
+ $ensure = present,
+) {
+ validate_ensure($ensure)
+ validate_slength($cert, 255)
+
+ file { '/etc/phaste.conf':
+ ensure => $ensure,
+ content => ordered_json({user => $user, cert => $cert, phab => $phab}),
+ owner => 'root',
+ group => 'root',
+ mode => '0444',
+ }
+
+ file { '/usr/local/bin/phaste':
+ ensure => $ensure,
+ source => 'puppet:///modules/phaste/phaste',
+ owner => 'root',
+ group => 'root',
+ mode => '0555',
+ require => File['/etc/phaste.conf'],
+ }
+}
--
To view, visit https://gerrit.wikimedia.org/r/174297
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4705178445e4f1437cb606f92f2108755c095e87
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ori.livneh <***@wikimedia.org>

[Ori.livneh (Code Review)]

Ori.livneh has submitted this change and it was merged.

Change subject: Add 'phaste' tool for pastebinning text to Phabricator
......................................................................


Add 'phaste' tool for pastebinning text to Phabricator

Provides the ability to pastebin text on Wikimedia's Phabricator installation
from anywhere in production by running 'phaste' -- e.g.:

# read from stdin:
grep xfs /var/log/syslog | phaste

# read from file:
phaste /var/log/upstart/service.log

Change-Id: I4705178445e4f1437cb606f92f2108755c095e87
---
A modules/base/files/phaste
M modules/base/manifests/init.pp
A modules/base/manifests/phaste.pp
3 files changed, 102 insertions(+), 0 deletions(-)

Approvals:
Ori.livneh: Looks good to me, approved
Rush: Looks good to me, but someone else must approve
jenkins-bot: Verified



diff --git a/modules/base/files/phaste b/modules/base/files/phaste
new file mode 100755
index 0000000..3d1f78b
--- /dev/null
+++ b/modules/base/files/phaste
@@ -0,0 +1,70 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+ phaste: Phabricator paste tool
+
+"""
+import sys
+reload(sys)
+sys.setdefaultencoding('utf-8')
+
+import fileinput
+import hashlib
+import json
+import time
+import urllib
+import urllib2
+
+
+class Conduit(object):
+ """A wrapper around Phabricator's Conduit API."""
+
+ def __init__(self, phab, user, cert):
+ self.phab = phab
+ self.user = user
+ self.cert = cert
+ self.credentials = None
+
+ def _do_request(self, method, data):
+ url = '%s/api/%s' % (self.phab, method)
+ req = urllib2.urlopen(url, data=urllib.urlencode(data))
+ resp = json.load(req)
+ if resp.get('error_info'):
+ raise RuntimeError('%(error_code)s: %(error_info)s' % resp)
+ return resp['result']
+
+ def _get_credentials(self):
+ if self.credentials is None:
+ token = int(time.time())
+ signature = hashlib.sha1(str(token) + self.cert).hexdigest()
+ params = {
+ 'client': 'phaste',
+ 'clientVersion': 0,
+ 'user': self.user,
+ 'authToken': token,
+ 'authSignature': signature,
+ }
+ data = {
+ 'params': json.dumps(params),
+ 'output': 'json',
+ '__conduit__': 'true',
+ }
+ self.credentials = self._do_request('conduit.connect', data)
+ return self.credentials
+
+ def call(self, method, **params):
+ """Call the conduit API."""
+ params['__conduit__'] = self._get_credentials()
+ data = {
+ 'params': json.dumps(params),
+ 'output': 'json',
+ }
+ return self._do_request(method, data)
+
+
+with open('/etc/phaste.conf') as f:
+ config = json.load(f)
+
+p = Conduit(phab=config['phab'], user=config['user'], cert=config['cert'])
+res = p.call('paste.create', content=''.join(fileinput.input()))
+print res['uri']
diff --git a/modules/base/manifests/init.pp b/modules/base/manifests/init.pp
index ab83082..c6633a4 100644
--- a/modules/base/manifests/init.pp
+++ b/modules/base/manifests/init.pp
@@ -230,6 +230,7 @@
base::motd,
base::standard-packages,
base::environment,
+ base::phaste,
base::platform,
base::screenconfig,
ssh::client,
diff --git a/modules/base/manifests/phaste.pp b/modules/base/manifests/phaste.pp
new file mode 100644
index 0000000..2b9b56c
--- /dev/null
+++ b/modules/base/manifests/phaste.pp
@@ -0,0 +1,31 @@
+# == Class: base::phaste
+#
+# Provisions 'phaste', a simple command-line tool for pastebinning text
+# onto Phabricator.
+#
+class base::phaste( $ensure = present ) {
+ include passwords::phabricator
+
+ $conf = {
+ user => 'ProdPasteBot',
+ cert => $passwords::phabricator::pastebot_cert,
+ phab => 'https://phabricator.wikimedia.org',
+ }
+
+ file { '/etc/phaste.conf':
+ ensure => $ensure,
+ content => ordered_json($conf),
+ owner => 'root',
+ group => 'root',
+ mode => '0444',
+ }
+
+ file { '/usr/local/bin/phaste':
+ ensure => $ensure,
+ source => 'puppet:///modules/base/phaste',
+ owner => 'root',
+ group => 'root',
+ mode => '0555',
+ require => File['/etc/phaste.conf'],
+ }
+}
--
To view, visit https://gerrit.wikimedia.org/r/174297
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I4705178445e4f1437cb606f92f2108755c095e87
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ori.livneh <***@wikimedia.org>
Gerrit-Reviewer: 20after4 <***@wikimedia.org>
Gerrit-Reviewer: Ori.livneh <***@wikimedia.org>
Gerrit-Reviewer: Rush <***@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>